2023-10-18T17:42:39.812 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-18T17:42:39.814 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}} --archive /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161 --owner scheduled_vshankar@teuthology --name vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi -- /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/orig.config.yaml 2023-10-18T17:42:39.861 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T17:42:39.937 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161 branch: wip-vshankar-testing-pacific-20231016.091648 description: fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}} email: vshankar@redhat.com first_in_suite: false job_id: '7429161' kernel: client: branch: testing flavor: default kdb: true sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-vshankar-testing-pacific-20231016.091648 ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: false mds op complaint time: 180 mds verify scatter: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - Replacing daemon mds sha1: b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-pacific-20231016.091648 sha1: b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 7825 sha1: b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 sleep_before_teardown: 0 subset: 33/1024 suite: fs suite_branch: wip-vshankar-testing-pacific-20231016.091648 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 targets: smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - mds_thrash: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_12:48:28 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-18T17:42:39.937 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa; will attempt to use it 2023-10-18T17:42:39.938 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks 2023-10-18T17:42:39.939 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-18T17:42:39.940 INFO:teuthology.task.internal:Checking packages... 2023-10-18T17:42:39.964 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5' 2023-10-18T17:42:39.964 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-18T17:42:39.965 INFO:teuthology.packaging:ref: None 2023-10-18T17:42:39.965 INFO:teuthology.packaging:tag: None 2023-10-18T17:42:39.965 INFO:teuthology.packaging:branch: wip-vshankar-testing-pacific-20231016.091648 2023-10-18T17:42:39.966 INFO:teuthology.packaging:sha1: b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 2023-10-18T17:42:39.966 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-vshankar-testing-pacific-20231016.091648 2023-10-18T17:42:40.198 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-100-gb4ddd52c-1focal 2023-10-18T17:42:40.199 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-18T17:42:40.208 INFO:teuthology.task.internal:no buildpackages task found 2023-10-18T17:42:40.208 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-18T17:42:40.245 INFO:teuthology.task.internal:Saving configuration 2023-10-18T17:42:40.262 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-18T17:42:40.273 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T17:42:40.298 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 17:34:57.932941', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-18T17:42:40.323 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 17:34:57.934678', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-18T17:42:40.323 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-18T17:42:40.331 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-10-18T17:42:40.332 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-10-18T17:42:40.332 INFO:teuthology.run_tasks:Running task console_log... 2023-10-18T17:42:40.409 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc6ed98ba60>, signals=[15]) 2023-10-18T17:42:40.411 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-18T17:42:40.420 INFO:teuthology.task.internal:Opening connections... 2023-10-18T17:42:40.420 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T17:42:40.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T17:42:40.469 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T17:42:40.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T17:42:40.517 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-18T17:42:40.529 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2023-10-18T17:42:40.546 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2023-10-18T17:42:40.546 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2023-10-18T17:42:40.594 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Ubuntu" 2023-10-18T17:42:40.594 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-18T17:42:40.594 INFO:teuthology.orchestra.run.smithi072.stdout:ID=ubuntu 2023-10-18T17:42:40.595 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE=debian 2023-10-18T17:42:40.595 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-18T17:42:40.595 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="20.04" 2023-10-18T17:42:40.595 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-18T17:42:40.595 INFO:teuthology.orchestra.run.smithi072.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-18T17:42:40.595 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-18T17:42:40.596 INFO:teuthology.orchestra.run.smithi072.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-18T17:42:40.596 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_CODENAME=focal 2023-10-18T17:42:40.596 INFO:teuthology.orchestra.run.smithi072.stdout:UBUNTU_CODENAME=focal 2023-10-18T17:42:40.597 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-10-18T17:42:40.620 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2023-10-18T17:42:40.627 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2023-10-18T17:42:40.628 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2023-10-18T17:42:40.675 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="Ubuntu" 2023-10-18T17:42:40.675 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-18T17:42:40.675 INFO:teuthology.orchestra.run.smithi179.stdout:ID=ubuntu 2023-10-18T17:42:40.675 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE=debian 2023-10-18T17:42:40.676 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-18T17:42:40.676 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="20.04" 2023-10-18T17:42:40.676 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-18T17:42:40.676 INFO:teuthology.orchestra.run.smithi179.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-18T17:42:40.676 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-18T17:42:40.676 INFO:teuthology.orchestra.run.smithi179.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-18T17:42:40.677 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_CODENAME=focal 2023-10-18T17:42:40.677 INFO:teuthology.orchestra.run.smithi179.stdout:UBUNTU_CODENAME=focal 2023-10-18T17:42:40.677 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2023-10-18T17:42:40.703 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-18T17:42:40.717 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-18T17:42:40.728 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-18T17:42:40.729 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2023-10-18T17:42:40.731 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2023-10-18T17:42:40.737 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-18T17:42:40.748 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-18T17:42:40.748 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2023-10-18T17:42:40.778 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2023-10-18T17:42:40.802 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-18T17:42:40.860 INFO:teuthology.run_tasks:Running task kernel... 2023-10-18T17:42:40.881 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'} 2023-10-18T17:42:40.882 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-18T17:42:40.883 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-18T17:42:40.883 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T17:42:40.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:42:40.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2023-10-18T17:42:41.134 DEBUG:teuthology.task.kernel:sha1 for client.0 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T17:42:41.135 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-18T17:42:41.135 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-10-18T17:42:41.142 INFO:teuthology.orchestra.run.smithi072.stdout:5.4.0-124-generic 2023-10-18T17:42:41.142 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T17:42:41.143 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-18T17:42:41.143 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-10-18T17:42:41.144 DEBUG:teuthology.orchestra.run.smithi072:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-18T17:42:41.206 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-10-18T17:42:41.363 INFO:teuthology.orchestra.run.smithi072.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-18T17:42:41.366 DEBUG:teuthology.orchestra.run.smithi072:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-18T17:42:41.395 DEBUG:teuthology.orchestra.run.smithi072:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-18T17:42:41.412 DEBUG:teuthology.orchestra.run.smithi072:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-18T17:43:32.256 INFO:teuthology.orchestra.run.smithi072.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-18T17:43:32.258 INFO:teuthology.orchestra.run.smithi072.stderr: * [new branch] main -> origin/main 2023-10-18T17:43:32.319 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20190312 -> 20190312 2023-10-18T17:43:32.319 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20190416 -> 20190416 2023-10-18T17:43:32.320 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20190514 -> 20190514 2023-10-18T17:43:32.320 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20190618 -> 20190618 2023-10-18T17:43:32.320 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20190717 -> 20190717 2023-10-18T17:43:32.320 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20190815 -> 20190815 2023-10-18T17:43:32.320 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20190923 -> 20190923 2023-10-18T17:43:32.321 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20191022 -> 20191022 2023-10-18T17:43:32.321 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20191215 -> 20191215 2023-10-18T17:43:32.321 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20200122 -> 20200122 2023-10-18T17:43:32.321 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20200316 -> 20200316 2023-10-18T17:43:32.321 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20200421 -> 20200421 2023-10-18T17:43:32.322 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20200519 -> 20200519 2023-10-18T17:43:32.322 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20200619 -> 20200619 2023-10-18T17:43:32.322 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20200721 -> 20200721 2023-10-18T17:43:32.322 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20200817 -> 20200817 2023-10-18T17:43:32.322 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20200918 -> 20200918 2023-10-18T17:43:32.323 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20201022 -> 20201022 2023-10-18T17:43:32.323 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20201118 -> 20201118 2023-10-18T17:43:32.323 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20201218 -> 20201218 2023-10-18T17:43:32.323 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20210208 -> 20210208 2023-10-18T17:43:32.323 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20210315 -> 20210315 2023-10-18T17:43:32.324 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20210511 -> 20210511 2023-10-18T17:43:32.324 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20210716 -> 20210716 2023-10-18T17:43:32.324 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20210818 -> 20210818 2023-10-18T17:43:32.324 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20210919 -> 20210919 2023-10-18T17:43:32.324 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20211027 -> 20211027 2023-10-18T17:43:32.325 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20211216 -> 20211216 2023-10-18T17:43:32.325 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20220209 -> 20220209 2023-10-18T17:43:32.325 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20220310 -> 20220310 2023-10-18T17:43:32.327 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20220411 -> 20220411 2023-10-18T17:43:32.327 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20220509 -> 20220509 2023-10-18T17:43:32.327 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20220610 -> 20220610 2023-10-18T17:43:32.328 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20220708 -> 20220708 2023-10-18T17:43:32.328 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20220815 -> 20220815 2023-10-18T17:43:32.328 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20220913 -> 20220913 2023-10-18T17:43:32.328 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20221012 -> 20221012 2023-10-18T17:43:32.329 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20221109 -> 20221109 2023-10-18T17:43:32.329 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20221214 -> 20221214 2023-10-18T17:43:32.329 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20230117 -> 20230117 2023-10-18T17:43:32.329 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20230210 -> 20230210 2023-10-18T17:43:32.329 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20230310 -> 20230310 2023-10-18T17:43:32.329 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20230404 -> 20230404 2023-10-18T17:43:32.330 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20230515 -> 20230515 2023-10-18T17:43:32.330 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20230625 -> 20230625 2023-10-18T17:43:32.330 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20230804 -> 20230804 2023-10-18T17:43:32.330 INFO:teuthology.orchestra.run.smithi072.stderr: * [new tag] 20230919 -> 20230919 2023-10-18T17:43:35.961 INFO:teuthology.orchestra.run.smithi072.stderr:Updating files: 52% (1659/3161) Updating files: 53% (1676/3161) Updating files: 54% (1707/3161) Updating files: 55% (1739/3161) Updating files: 56% (1771/3161) Updating files: 57% (1802/3161) Updating files: 58% (1834/3161) Updating files: 59% (1865/3161) Updating files: 60% (1897/3161) Updating files: 61% (1929/3161) Updating files: 62% (1960/3161) Updating files: 63% (1992/3161) Updating files: 64% (2024/3161) Updating files: 65% (2055/3161) Updating files: 66% (2087/3161) Updating files: 66% (2108/3161) Updating files: 67% (2118/3161) Updating files: 68% (2150/3161) Updating files: 69% (2182/3161) Updating files: 70% (2213/3161) Updating files: 71% (2245/3161) Updating files: 72% (2276/3161) Updating files: 73% (2308/3161) Updating files: 74% (2340/3161) Updating files: 75% (2371/3161) Updating files: 76% (2403/3161) Updating files: 77% (2434/3161) Updating files: 78% (2466/3161) Updating files: 79% (2498/3161) Updating files: 80% (2529/3161) Updating files: 81% (2561/3161) Updating files: 81% (2591/3161) Updating files: 82% (2593/3161) Updating files: 83% (2624/3161) Updating files: 84% (2656/3161) Updating files: 85% (2687/3161) Updating files: 86% (2719/3161) Updating files: 87% (2751/3161) Updating files: 88% (2782/3161) Updating files: 89% (2814/3161) Updating files: 90% (2845/3161) Updating files: 91% (2877/3161) Updating files: 92% (2909/3161) Updating files: 93% (2940/3161) Updating files: 94% (2972/3161) Updating files: 95% (3003/3161) Updating files: 96% (3035/3161) Updating files: 97% (3067/3161) Updating files: 98% (3098/3161) Updating files: 99% (3130/3161) Updating files: 100% (3161/3161) Updating files: 100% (3161/3161), done. 2023-10-18T17:43:35.966 INFO:teuthology.orchestra.run.smithi072.stdout:HEAD is now at 8ff933de Merge branch 'robot/pr-0-1697570762' into 'main' 2023-10-18T17:43:35.982 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T17:43:35.983 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T17:43:36.222 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ 2023-10-18T17:43:36.223 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/ --input-file=- 2023-10-18T17:43:36.225 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi072.front.sepia.ceph.com to complete... 2023-10-18T17:43:37.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18 17:43:37 URL:https://4.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g47672119149b/linux-image-6.6.0-rc5-g47672119149b_6.6.0-rc5-g47672119149b-1_amd64.deb [38904644/38904644] -> "/tmp/linux-image.deb" [1] 2023-10-18T17:43:37.808 INFO:teuthology.orchestra.run.smithi072.stderr:FINISHED --2023-10-18 17:43:37-- 2023-10-18T17:43:37.808 INFO:teuthology.orchestra.run.smithi072.stderr:Total wall clock time: 1.5s 2023-10-18T17:43:37.808 INFO:teuthology.orchestra.run.smithi072.stderr:Downloaded: 1 files, 37M in 1.4s (27.0 MB/s) 2023-10-18T17:43:37.809 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T17:43:37.810 DEBUG:teuthology.orchestra.run.smithi072:> sudo dpkg -i /tmp/linux-image.deb 2023-10-18T17:43:38.126 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g47672119149b. 2023-10-18T17:43:40.711 INFO:teuthology.orchestra.run.smithi072.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-10-18T17:43:40.716 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-18T17:43:40.846 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-18T17:43:44.332 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up linux-image-6.6.0-rc5-g47672119149b (6.6.0-rc5-g47672119149b-1) ... 2023-10-18T17:43:44.454 INFO:teuthology.orchestra.run.smithi072.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-18T17:44:16.911 INFO:teuthology.orchestra.run.smithi072.stderr:Sourcing file `/etc/default/grub' 2023-10-18T17:44:16.923 INFO:teuthology.orchestra.run.smithi072.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-18T17:44:16.942 INFO:teuthology.orchestra.run.smithi072.stderr:Generating grub configuration file ... 2023-10-18T17:44:17.322 INFO:teuthology.orchestra.run.smithi072.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T17:44:17.361 INFO:teuthology.orchestra.run.smithi072.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-18T17:44:17.735 INFO:teuthology.orchestra.run.smithi072.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-18T17:44:17.748 INFO:teuthology.orchestra.run.smithi072.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-18T17:44:20.814 INFO:teuthology.orchestra.run.smithi072.stderr:done 2023-10-18T17:44:20.999 DEBUG:teuthology.orchestra.run.smithi072:> dpkg-deb -c /tmp/linux-image.deb 2023-10-18T17:44:21.008 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./ 2023-10-18T17:44:21.009 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./boot/ 2023-10-18T17:44:21.009 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 4415097 2023-10-10 19:59 ./boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T17:44:21.022 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 169298 2023-10-10 19:59 ./boot/config-6.6.0-rc5-g47672119149b 2023-10-18T17:44:21.022 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11352928 2023-10-10 19:59 ./boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T17:44:21.044 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./etc/ 2023-10-18T17:44:21.044 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./etc/kernel/ 2023-10-18T17:44:21.044 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./etc/kernel/postinst.d/ 2023-10-18T17:44:21.044 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./etc/kernel/postrm.d/ 2023-10-18T17:44:21.045 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./etc/kernel/preinst.d/ 2023-10-18T17:44:21.045 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./etc/kernel/prerm.d/ 2023-10-18T17:44:21.045 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/ 2023-10-18T17:44:21.045 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/ 2023-10-18T17:44:21.046 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/ 2023-10-18T17:44:21.046 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/ 2023-10-18T17:44:21.046 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/ 2023-10-18T17:44:21.046 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/ 2023-10-18T17:44:21.046 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ 2023-10-18T17:44:21.047 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 490233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-18T17:44:21.049 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-18T17:44:21.049 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-18T17:44:21.050 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-18T17:44:21.050 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 93113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-18T17:44:21.050 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 106729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-18T17:44:21.050 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 152241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-18T17:44:21.050 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-18T17:44:21.051 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-18T17:44:21.051 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 83889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-18T17:44:21.051 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-18T17:44:21.051 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-18T17:44:21.052 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-18T17:44:21.052 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-18T17:44:21.052 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-18T17:44:21.052 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-18T17:44:21.052 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 58585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-18T17:44:21.052 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-18T17:44:21.053 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-18T17:44:21.053 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-18T17:44:21.053 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/ 2023-10-18T17:44:21.053 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/ 2023-10-18T17:44:21.054 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-18T17:44:21.054 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-18T17:44:21.054 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpuid.ko 2023-10-18T17:44:21.054 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/msr.ko 2023-10-18T17:44:21.054 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/ 2023-10-18T17:44:21.055 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 353761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-18T17:44:21.055 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 881713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-18T17:44:21.055 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 2276817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-18T17:44:21.113 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ 2023-10-18T17:44:21.113 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/af_alg.ko 2023-10-18T17:44:21.114 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_hash.ko 2023-10-18T17:44:21.114 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/algif_skcipher.ko 2023-10-18T17:44:21.114 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-18T17:44:21.114 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19641 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-18T17:44:21.115 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-18T17:44:21.115 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/ 2023-10-18T17:44:21.115 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-18T17:44:21.115 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/ 2023-10-18T17:44:21.115 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-18T17:44:21.116 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-18T17:44:21.116 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-18T17:44:21.116 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-18T17:44:21.116 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-18T17:44:21.116 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-18T17:44:21.117 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-18T17:44:21.117 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22641 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-18T17:44:21.117 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-18T17:44:21.117 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-18T17:44:21.117 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-18T17:44:21.118 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-18T17:44:21.118 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-18T17:44:21.118 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-18T17:44:21.118 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-18T17:44:21.118 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-18T17:44:21.119 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-18T17:44:21.119 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-18T17:44:21.119 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cryptd.ko 2023-10-18T17:44:21.119 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-18T17:44:21.119 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_simd.ko 2023-10-18T17:44:21.120 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-18T17:44:21.120 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-18T17:44:21.120 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-18T17:44:21.120 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-18T17:44:21.120 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-18T17:44:21.121 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-18T17:44:21.121 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-18T17:44:21.121 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-18T17:44:21.121 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lrw.ko 2023-10-18T17:44:21.121 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-18T17:44:21.121 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-18T17:44:21.122 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-18T17:44:21.122 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-18T17:44:21.122 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-18T17:44:21.122 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-18T17:44:21.123 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-18T17:44:21.123 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-18T17:44:21.123 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-18T17:44:21.123 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 149193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-18T17:44:21.123 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-18T17:44:21.123 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-18T17:44:21.124 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-18T17:44:21.124 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-18T17:44:21.124 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-18T17:44:21.124 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-18T17:44:21.124 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-18T17:44:21.124 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xts.ko 2023-10-18T17:44:21.125 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-18T17:44:21.125 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ 2023-10-18T17:44:21.125 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ 2023-10-18T17:44:21.125 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-18T17:44:21.125 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-18T17:44:21.126 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-18T17:44:21.126 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/ 2023-10-18T17:44:21.126 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/apei/einj.ko 2023-10-18T17:44:21.126 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-18T17:44:21.126 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-18T17:44:21.127 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-18T17:44:21.127 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-18T17:44:21.127 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 103025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-18T17:44:21.127 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ 2023-10-18T17:44:21.127 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-18T17:44:21.127 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 88865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci.ko 2023-10-18T17:44:21.128 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-18T17:44:21.128 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 88289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci.ko 2023-10-18T17:44:21.128 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-18T17:44:21.128 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-18T17:44:21.128 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-18T17:44:21.129 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-18T17:44:21.129 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-18T17:44:21.129 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-18T17:44:21.129 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-18T17:44:21.129 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-18T17:44:21.129 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-18T17:44:21.130 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-18T17:44:21.130 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-18T17:44:21.130 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-18T17:44:21.130 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-18T17:44:21.130 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-18T17:44:21.131 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-18T17:44:21.131 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-18T17:44:21.131 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-18T17:44:21.131 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-18T17:44:21.131 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_legacy.ko 2023-10-18T17:44:21.132 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-18T17:44:21.132 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-18T17:44:21.132 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-18T17:44:21.132 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-18T17:44:21.132 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-18T17:44:21.133 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-18T17:44:21.133 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-18T17:44:21.133 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-18T17:44:21.133 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-18T17:44:21.133 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-18T17:44:21.133 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-18T17:44:21.134 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-18T17:44:21.134 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-18T17:44:21.134 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_radisys.ko 2023-10-18T17:44:21.134 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-18T17:44:21.134 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rz1000.ko 2023-10-18T17:44:21.135 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-18T17:44:21.135 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-18T17:44:21.135 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-18T17:44:21.135 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-18T17:44:21.135 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-18T17:44:21.135 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-18T17:44:21.136 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-18T17:44:21.136 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-18T17:44:21.136 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 83345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-18T17:44:21.136 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 70393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-18T17:44:21.136 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37377 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-18T17:44:21.137 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-18T17:44:21.137 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-18T17:44:21.137 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-18T17:44:21.137 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-18T17:44:21.137 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-18T17:44:21.137 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-18T17:44:21.138 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-18T17:44:21.138 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-18T17:44:21.138 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-18T17:44:21.138 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ 2023-10-18T17:44:21.138 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-18T17:44:21.139 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/ 2023-10-18T17:44:21.139 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 124473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-18T17:44:21.139 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/ 2023-10-18T17:44:21.139 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/ 2023-10-18T17:44:21.139 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 113665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-18T17:44:21.140 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/ 2023-10-18T17:44:21.140 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 965585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-18T17:44:21.141 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 173225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-18T17:44:21.141 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/ 2023-10-18T17:44:21.141 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 94297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-18T17:44:21.142 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 115793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-18T17:44:21.142 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/ 2023-10-18T17:44:21.142 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 110449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-18T17:44:21.142 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 98193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-18T17:44:21.143 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 263769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-18T17:44:21.143 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/ 2023-10-18T17:44:21.143 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 98145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-18T17:44:21.143 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/ 2023-10-18T17:44:21.143 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60217 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-18T17:44:21.144 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ 2023-10-18T17:44:21.144 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30705 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/applicom.ko 2023-10-18T17:44:21.144 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-18T17:44:21.144 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/ 2023-10-18T17:44:21.144 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-18T17:44:21.145 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-18T17:44:21.145 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-18T17:44:21.145 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-18T17:44:21.145 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-18T17:44:21.145 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ 2023-10-18T17:44:21.146 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-18T17:44:21.146 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 148601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-18T17:44:21.146 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-18T17:44:21.146 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 139313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-18T17:44:21.146 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 70921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-18T17:44:21.147 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-18T17:44:21.147 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-18T17:44:21.147 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/ 2023-10-18T17:44:21.147 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 84097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-18T17:44:21.147 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/nvram.ko 2023-10-18T17:44:21.147 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-18T17:44:21.148 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-18T17:44:21.148 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/ 2023-10-18T17:44:21.148 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-18T17:44:21.148 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-18T17:44:21.148 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-18T17:44:21.149 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-18T17:44:21.149 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-18T17:44:21.149 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-18T17:44:21.149 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-18T17:44:21.149 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/ 2023-10-18T17:44:21.149 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-18T17:44:21.150 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-18T17:44:21.150 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-18T17:44:21.150 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/ 2023-10-18T17:44:21.150 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-palmas.ko 2023-10-18T17:44:21.150 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-si5351.ko 2023-10-18T17:44:21.151 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-twl6040.ko 2023-10-18T17:44:21.151 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/clk/clk-wm831x.ko 2023-10-18T17:44:21.151 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/ 2023-10-18T17:44:21.151 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-18T17:44:21.151 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-18T17:44:21.152 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-18T17:44:21.152 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ 2023-10-18T17:44:21.152 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ 2023-10-18T17:44:21.152 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 70801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-18T17:44:21.152 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 243025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-18T17:44:21.153 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/ 2023-10-18T17:44:21.153 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/ 2023-10-18T17:44:21.153 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-18T17:44:21.153 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 438761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-18T17:44:21.153 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-18T17:44:21.154 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-18T17:44:21.154 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-18T17:44:21.154 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-18T17:44:21.154 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/ 2023-10-18T17:44:21.154 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 68857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-18T17:44:21.155 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/ 2023-10-18T17:44:21.155 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-18T17:44:21.155 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ 2023-10-18T17:44:21.155 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/ 2023-10-18T17:44:21.155 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-18T17:44:21.156 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ 2023-10-18T17:44:21.156 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 122569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-18T17:44:21.156 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ 2023-10-18T17:44:21.156 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 79953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-18T17:44:21.156 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-18T17:44:21.157 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 63617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-18T17:44:21.157 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-18T17:44:21.157 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-18T17:44:21.157 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-18T17:44:21.157 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-18T17:44:21.158 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-18T17:44:21.158 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 52041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-18T17:44:21.158 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-18T17:44:21.158 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-18T17:44:21.158 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 56825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-18T17:44:21.159 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-18T17:44:21.159 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/ 2023-10-18T17:44:21.159 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-18T17:44:21.159 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-18T17:44:21.159 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-18T17:44:21.160 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-18T17:44:21.160 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-18T17:44:21.160 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-18T17:44:21.160 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-18T17:44:21.160 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/ 2023-10-18T17:44:21.160 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-18T17:44:21.161 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/ 2023-10-18T17:44:21.161 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-18T17:44:21.161 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-18T17:44:21.161 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/ 2023-10-18T17:44:21.161 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-18T17:44:21.162 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-18T17:44:21.162 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-18T17:44:21.162 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-18T17:44:21.162 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-18T17:44:21.162 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-18T17:44:21.163 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 77849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-18T17:44:21.163 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-generic.ko 2023-10-18T17:44:21.163 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-18T17:44:21.163 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-18T17:44:21.163 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-18T17:44:21.164 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-18T17:44:21.164 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-18T17:44:21.164 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-18T17:44:21.164 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-18T17:44:21.164 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-18T17:44:21.165 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-18T17:44:21.165 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-18T17:44:21.165 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14377 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-18T17:44:21.165 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch.ko 2023-10-18T17:44:21.165 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-18T17:44:21.166 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-18T17:44:21.166 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-18T17:44:21.166 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-18T17:44:21.166 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-18T17:44:21.166 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15377 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-18T17:44:21.166 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16281 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-18T17:44:21.167 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-18T17:44:21.167 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-18T17:44:21.167 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/ 2023-10-18T17:44:21.167 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-a4tech.ko 2023-10-18T17:44:21.167 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-apple.ko 2023-10-18T17:44:21.168 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-18T17:44:21.168 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-18T17:44:21.168 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-18T17:44:21.168 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-belkin.ko 2023-10-18T17:44:21.168 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cherry.ko 2023-10-18T17:44:21.169 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-chicony.ko 2023-10-18T17:44:21.169 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cp2112.ko 2023-10-18T17:44:21.169 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-cypress.ko 2023-10-18T17:44:21.169 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-18T17:44:21.169 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-18T17:44:21.170 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-18T17:44:21.170 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-18T17:44:21.170 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ezkey.ko 2023-10-18T17:44:21.170 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-18T17:44:21.170 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-generic.ko 2023-10-18T17:44:21.171 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-18T17:44:21.171 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-18T17:44:21.171 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-18T17:44:21.171 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-18T17:44:21.171 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-18T17:44:21.172 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-18T17:44:21.172 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-18T17:44:21.172 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kensington.ko 2023-10-18T17:44:21.172 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-18T17:44:21.172 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-18T17:44:21.173 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-18T17:44:21.173 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-18T17:44:21.173 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42361 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-18T17:44:21.173 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-18T17:44:21.173 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 58185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-18T17:44:21.174 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 101369 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-18T17:44:21.174 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 80993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech.ko 2023-10-18T17:44:21.174 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-18T17:44:21.174 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-microsoft.ko 2023-10-18T17:44:21.174 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10361 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-monterey.ko 2023-10-18T17:44:21.175 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-18T17:44:21.175 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ntrig.ko 2023-10-18T17:44:21.175 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-18T17:44:21.175 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-18T17:44:21.175 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-18T17:44:21.176 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 72937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-18T17:44:21.176 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-18T17:44:21.176 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-18T17:44:21.176 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-18T17:44:21.176 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-18T17:44:21.177 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-18T17:44:21.177 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-18T17:44:21.177 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-18T17:44:21.177 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-18T17:44:21.177 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-18T17:44:21.177 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-18T17:44:21.178 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-18T17:44:21.178 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-18T17:44:21.178 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-18T17:44:21.178 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-18T17:44:21.178 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-18T17:44:21.179 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-18T17:44:21.179 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-18T17:44:21.179 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-18T17:44:21.179 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-18T17:44:21.179 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-18T17:44:21.180 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-18T17:44:21.180 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-18T17:44:21.180 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-18T17:44:21.180 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-18T17:44:21.180 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-18T17:44:21.181 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-18T17:44:21.181 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-18T17:44:21.181 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-18T17:44:21.181 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-18T17:44:21.181 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 71073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-18T17:44:21.182 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-18T17:44:21.182 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 143473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-18T17:44:21.182 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-18T17:44:21.182 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-18T17:44:21.182 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-18T17:44:21.183 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 267825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid.ko 2023-10-18T17:44:21.183 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-18T17:44:21.183 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/ 2023-10-18T17:44:21.183 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 125721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-18T17:44:21.183 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-18T17:44:21.184 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-18T17:44:21.184 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 217873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-18T17:44:21.184 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/ 2023-10-18T17:44:21.184 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 64201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-18T17:44:21.184 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 89625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-18T17:44:21.185 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 305465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-18T17:44:21.185 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ 2023-10-18T17:44:21.185 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-18T17:44:21.185 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 61393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-18T17:44:21.185 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-18T17:44:21.186 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-18T17:44:21.186 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-18T17:44:21.186 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-18T17:44:21.186 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-18T17:44:21.186 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 97977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-18T17:44:21.187 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-18T17:44:21.187 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 45553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-18T17:44:21.187 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-18T17:44:21.187 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-18T17:44:21.187 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-18T17:44:21.188 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-18T17:44:21.188 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 74433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-18T17:44:21.188 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-18T17:44:21.188 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-18T17:44:21.188 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-18T17:44:21.189 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-18T17:44:21.189 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 55265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-18T17:44:21.189 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-18T17:44:21.189 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 58617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-18T17:44:21.189 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-18T17:44:21.190 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-18T17:44:21.190 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28281 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-18T17:44:21.190 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-18T17:44:21.190 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-18T17:44:21.190 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-18T17:44:21.191 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 99601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-18T17:44:21.191 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-18T17:44:21.191 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-18T17:44:21.191 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-18T17:44:21.191 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc2103.ko 2023-10-18T17:44:21.192 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-18T17:44:21.192 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-18T17:44:21.192 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 94457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-18T17:44:21.192 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-18T17:44:21.192 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-18T17:44:21.192 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 55449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-18T17:44:21.193 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-18T17:44:21.193 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-18T17:44:21.193 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 46801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-18T17:44:21.193 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 45529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-18T17:44:21.193 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hih6130.ko 2023-10-18T17:44:21.194 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-18T17:44:21.194 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-18T17:44:21.194 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-18T17:44:21.194 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-18T17:44:21.194 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-18T17:44:21.195 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-18T17:44:21.195 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-18T17:44:21.195 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 130449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-18T17:44:21.195 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/jc42.ko 2023-10-18T17:44:21.195 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-18T17:44:21.196 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-18T17:44:21.196 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-18T17:44:21.196 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 48601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-18T17:44:21.196 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-18T17:44:21.196 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-18T17:44:21.197 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-18T17:44:21.197 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-18T17:44:21.197 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-18T17:44:21.197 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-18T17:44:21.197 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 72833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-18T17:44:21.198 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 46153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-18T17:44:21.198 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-18T17:44:21.198 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-18T17:44:21.198 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 108841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-18T17:44:21.198 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-18T17:44:21.199 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-18T17:44:21.199 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-18T17:44:21.199 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-18T17:44:21.199 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-18T17:44:21.199 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-18T17:44:21.199 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-18T17:44:21.200 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-18T17:44:21.200 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-18T17:44:21.200 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-18T17:44:21.200 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-18T17:44:21.200 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-18T17:44:21.201 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-18T17:44:21.201 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-18T17:44:21.201 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-18T17:44:21.201 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-18T17:44:21.201 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-18T17:44:21.201 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-18T17:44:21.202 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-18T17:44:21.202 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-18T17:44:21.202 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-18T17:44:21.202 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 119817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-18T17:44:21.202 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60705 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-18T17:44:21.203 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 59737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-18T17:44:21.203 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 82337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-18T17:44:21.203 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-18T17:44:21.203 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-18T17:44:21.203 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ 2023-10-18T17:44:21.204 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-18T17:44:21.204 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-18T17:44:21.204 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-18T17:44:21.204 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-18T17:44:21.204 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-18T17:44:21.204 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-18T17:44:21.205 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-18T17:44:21.205 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 82577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-18T17:44:21.205 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-18T17:44:21.205 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-18T17:44:21.205 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-18T17:44:21.206 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-18T17:44:21.206 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-18T17:44:21.206 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18705 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-18T17:44:21.206 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-18T17:44:21.206 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-18T17:44:21.207 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-18T17:44:21.207 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-18T17:44:21.207 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-18T17:44:21.207 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-18T17:44:21.207 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-18T17:44:21.208 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-18T17:44:21.208 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-18T17:44:21.208 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-18T17:44:21.208 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-18T17:44:21.208 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-18T17:44:21.209 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-18T17:44:21.209 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-18T17:44:21.209 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-18T17:44:21.209 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-18T17:44:21.209 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 52457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-18T17:44:21.210 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-18T17:44:21.210 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 57513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-18T17:44:21.210 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-18T17:44:21.210 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 71345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-18T17:44:21.210 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 63289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-18T17:44:21.211 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 71321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-18T17:44:21.211 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 101473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-18T17:44:21.211 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 119505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-18T17:44:21.211 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-18T17:44:21.211 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-18T17:44:21.212 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-18T17:44:21.212 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-18T17:44:21.212 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/ 2023-10-18T17:44:21.212 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/ 2023-10-18T17:44:21.212 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-18T17:44:21.213 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-18T17:44:21.213 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/ 2023-10-18T17:44:21.213 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-18T17:44:21.213 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-18T17:44:21.213 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-18T17:44:21.214 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-18T17:44:21.214 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-18T17:44:21.214 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-18T17:44:21.214 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-18T17:44:21.214 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 6929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-18T17:44:21.214 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-18T17:44:21.215 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-18T17:44:21.215 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-18T17:44:21.215 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-18T17:44:21.215 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-18T17:44:21.215 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-18T17:44:21.216 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-18T17:44:21.216 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-18T17:44:21.216 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-18T17:44:21.216 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-18T17:44:21.216 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-18T17:44:21.217 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-18T17:44:21.217 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-18T17:44:21.217 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-18T17:44:21.217 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-18T17:44:21.217 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-18T17:44:21.218 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-18T17:44:21.218 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-18T17:44:21.218 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-18T17:44:21.218 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-18T17:44:21.218 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-18T17:44:21.219 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29361 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-18T17:44:21.219 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-18T17:44:21.219 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-18T17:44:21.219 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-18T17:44:21.219 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-18T17:44:21.219 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-18T17:44:21.220 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-18T17:44:21.220 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-18T17:44:21.220 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-18T17:44:21.220 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-18T17:44:21.220 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-18T17:44:21.221 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/ 2023-10-18T17:44:21.221 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-18T17:44:21.221 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-18T17:44:21.221 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-18T17:44:21.221 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ 2023-10-18T17:44:21.221 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ 2023-10-18T17:44:21.222 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 273609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-18T17:44:21.222 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 879969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-18T17:44:21.222 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 69337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-18T17:44:21.222 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 324217 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-18T17:44:21.222 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 97985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-18T17:44:21.223 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 282273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-18T17:44:21.223 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-18T17:44:21.223 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ 2023-10-18T17:44:21.223 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-18T17:44:21.223 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 493073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-18T17:44:21.224 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-18T17:44:21.224 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 419737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-18T17:44:21.224 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-18T17:44:21.224 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 739025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-18T17:44:21.224 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ 2023-10-18T17:44:21.224 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 296625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-18T17:44:21.225 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-18T17:44:21.225 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 153697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-18T17:44:21.225 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/ 2023-10-18T17:44:21.225 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 168969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-18T17:44:21.225 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ 2023-10-18T17:44:21.226 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-18T17:44:21.226 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 263081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-18T17:44:21.226 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ 2023-10-18T17:44:21.226 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 100241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-18T17:44:21.226 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ 2023-10-18T17:44:21.226 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 111425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-18T17:44:21.227 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ 2023-10-18T17:44:21.227 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 128753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-18T17:44:21.227 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-18T17:44:21.227 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 129529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-18T17:44:21.227 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ 2023-10-18T17:44:21.228 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/evbug.ko 2023-10-18T17:44:21.228 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/ff-memless.ko 2023-10-18T17:44:21.228 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/ 2023-10-18T17:44:21.228 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-18T17:44:21.228 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-18T17:44:21.228 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-18T17:44:21.229 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-18T17:44:21.229 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/ 2023-10-18T17:44:21.229 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-18T17:44:21.229 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-18T17:44:21.229 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-18T17:44:21.229 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-18T17:44:21.230 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-18T17:44:21.230 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-18T17:44:21.230 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-18T17:44:21.230 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-18T17:44:21.230 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-18T17:44:21.231 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-18T17:44:21.231 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-18T17:44:21.231 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-18T17:44:21.231 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-18T17:44:21.231 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-18T17:44:21.232 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-18T17:44:21.232 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-18T17:44:21.232 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-18T17:44:21.232 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-18T17:44:21.232 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-18T17:44:21.233 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-18T17:44:21.233 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-18T17:44:21.233 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-18T17:44:21.233 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-18T17:44:21.233 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-18T17:44:21.234 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/matrix-keymap.ko 2023-10-18T17:44:21.234 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/ 2023-10-18T17:44:21.234 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-18T17:44:21.234 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-18T17:44:21.234 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 123681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-18T17:44:21.235 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 89449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-18T17:44:21.235 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-18T17:44:21.235 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 335977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/psmouse.ko 2023-10-18T17:44:21.235 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-18T17:44:21.235 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-18T17:44:21.236 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-18T17:44:21.236 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-18T17:44:21.236 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/ 2023-10-18T17:44:21.236 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 109873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-18T17:44:21.236 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ 2023-10-18T17:44:21.237 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-18T17:44:21.237 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-18T17:44:21.237 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ct82c710.ko 2023-10-18T17:44:21.237 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-18T17:44:21.237 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/parkbd.ko 2023-10-18T17:44:21.238 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/pcips2.ko 2023-10-18T17:44:21.238 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/ps2mult.ko 2023-10-18T17:44:21.238 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-18T17:44:21.238 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serport.ko 2023-10-18T17:44:21.238 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-18T17:44:21.239 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/ 2023-10-18T17:44:21.239 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/ 2023-10-18T17:44:21.239 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-18T17:44:21.239 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/ 2023-10-18T17:44:21.239 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-18T17:44:21.240 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-adp5520.ko 2023-10-18T17:44:21.240 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 59753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-bd2802.ko 2023-10-18T17:44:21.240 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-18T17:44:21.240 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da903x.ko 2023-10-18T17:44:21.240 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-da9052.ko 2023-10-18T17:44:21.240 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-gpio.ko 2023-10-18T17:44:21.241 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-18T17:44:21.241 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3533.ko 2023-10-18T17:44:21.241 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm355x.ko 2023-10-18T17:44:21.241 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3642.ko 2023-10-18T17:44:21.241 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-18T17:44:21.242 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp8788.ko 2023-10-18T17:44:21.242 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-max8997.ko 2023-10-18T17:44:21.242 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-mc13783.ko 2023-10-18T17:44:21.242 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-18T17:44:21.242 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca9532.ko 2023-10-18T17:44:21.243 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca955x.ko 2023-10-18T17:44:21.243 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-pca963x.ko 2023-10-18T17:44:21.243 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-18T17:44:21.243 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-tca6507.ko 2023-10-18T17:44:21.243 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-18T17:44:21.244 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-wm8350.ko 2023-10-18T17:44:21.244 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ 2023-10-18T17:44:21.244 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-18T17:44:21.244 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-18T17:44:21.244 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-18T17:44:21.244 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-18T17:44:21.245 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-18T17:44:21.245 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-18T17:44:21.245 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-18T17:44:21.245 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-18T17:44:21.245 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/ 2023-10-18T17:44:21.246 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/ 2023-10-18T17:44:21.246 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 654969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-18T17:44:21.246 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-18T17:44:21.246 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 89601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bufio.ko 2023-10-18T17:44:21.246 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-18T17:44:21.247 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 156009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-18T17:44:21.247 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 106897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-18T17:44:21.247 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-18T17:44:21.247 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-era.ko 2023-10-18T17:44:21.247 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-18T17:44:21.248 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-18T17:44:21.248 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log.ko 2023-10-18T17:44:21.248 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-mirror.ko 2023-10-18T17:44:21.248 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 78033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-18T17:44:21.248 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-18T17:44:21.248 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 70825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-18T17:44:21.249 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-region-hash.ko 2023-10-18T17:44:21.249 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-18T17:44:21.249 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-18T17:44:21.249 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-snapshot.ko 2023-10-18T17:44:21.249 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-18T17:44:21.250 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 181865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-18T17:44:21.250 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-18T17:44:21.250 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-zero.ko 2023-10-18T17:44:21.250 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-18T17:44:21.250 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-18T17:44:21.251 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-18T17:44:21.251 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/ 2023-10-18T17:44:21.251 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 199937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-18T17:44:21.251 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-18T17:44:21.251 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 113073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-18T17:44:21.252 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 158681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-18T17:44:21.252 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 395721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-18T17:44:21.252 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/ 2023-10-18T17:44:21.252 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/ 2023-10-18T17:44:21.252 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 178137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-18T17:44:21.252 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 63385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-18T17:44:21.253 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-18T17:44:21.253 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-18T17:44:21.253 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 123321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-18T17:44:21.253 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 79041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-18T17:44:21.253 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-18T17:44:21.254 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ 2023-10-18T17:44:21.254 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm800.ko 2023-10-18T17:44:21.254 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm805.ko 2023-10-18T17:44:21.254 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/88pm80x.ko 2023-10-18T17:44:21.254 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-18T17:44:21.255 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 123721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/arizona.ko 2023-10-18T17:44:21.255 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/bcm590xx.ko 2023-10-18T17:44:21.255 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-18T17:44:21.255 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/dln2.ko 2023-10-18T17:44:21.255 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-18T17:44:21.256 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/kempld-core.ko 2023-10-18T17:44:21.256 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-core.ko 2023-10-18T17:44:21.256 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-18T17:44:21.256 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lp3943.ko 2023-10-18T17:44:21.256 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-18T17:44:21.256 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-18T17:44:21.257 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/max8907.ko 2023-10-18T17:44:21.257 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-18T17:44:21.257 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-18T17:44:21.257 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/menf21bmc.ko 2023-10-18T17:44:21.258 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-18T17:44:21.258 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-18T17:44:21.258 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/pcf50633.ko 2023-10-18T17:44:21.258 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-18T17:44:21.258 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/retu-mfd.ko 2023-10-18T17:44:21.258 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 80425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/si476x-core.ko 2023-10-18T17:44:21.259 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 48457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-18T17:44:21.259 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-18T17:44:21.259 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps65010.ko 2023-10-18T17:44:21.259 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/tps6507x.ko 2023-10-18T17:44:21.259 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-18T17:44:21.260 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-18T17:44:21.260 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-18T17:44:21.260 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wm8994.ko 2023-10-18T17:44:21.260 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ 2023-10-18T17:44:21.260 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-18T17:44:21.261 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 48489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-18T17:44:21.261 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/ 2023-10-18T17:44:21.261 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 46585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-18T17:44:21.261 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-18T17:44:21.261 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-18T17:44:21.261 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 45593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-18T17:44:21.262 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/ 2023-10-18T17:44:21.262 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-18T17:44:21.262 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35281 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/c2port/core.ko 2023-10-18T17:44:21.262 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/ 2023-10-18T17:44:21.262 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-18T17:44:21.263 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ds1682.ko 2023-10-18T17:44:21.263 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/dummy-irq.ko 2023-10-18T17:44:21.263 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/ 2023-10-18T17:44:21.263 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/echo/echo.ko 2023-10-18T17:44:21.263 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/ 2023-10-18T17:44:21.264 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-18T17:44:21.264 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-18T17:44:21.264 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-18T17:44:21.264 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-18T17:44:21.264 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-18T17:44:21.264 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/ 2023-10-18T17:44:21.265 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 152201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-18T17:44:21.265 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hmc6352.ko 2023-10-18T17:44:21.265 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-18T17:44:21.265 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ 2023-10-18T17:44:21.265 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 72129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-18T17:44:21.266 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ics932s401.ko 2023-10-18T17:44:21.266 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-18T17:44:21.266 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-18T17:44:21.266 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/ 2023-10-18T17:44:21.266 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-18T17:44:21.267 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-18T17:44:21.267 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/ 2023-10-18T17:44:21.267 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 125209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-18T17:44:21.267 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 56521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-18T17:44:21.267 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 310113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-18T17:44:21.268 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/phantom.ko 2023-10-18T17:44:21.268 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/ 2023-10-18T17:44:21.268 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 74265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-18T17:44:21.268 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-18T17:44:21.268 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-18T17:44:21.268 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-18T17:44:21.269 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 48697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-18T17:44:21.269 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/ 2023-10-18T17:44:21.269 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 183609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-18T17:44:21.269 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ 2023-10-18T17:44:21.269 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/ 2023-10-18T17:44:21.270 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 490129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-18T17:44:21.270 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/ 2023-10-18T17:44:21.270 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/ 2023-10-18T17:44:21.270 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 106033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-18T17:44:21.270 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-18T17:44:21.271 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-18T17:44:21.271 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-18T17:44:21.271 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 453737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-18T17:44:21.367 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-18T17:44:21.368 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-18T17:44:21.368 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ 2023-10-18T17:44:21.368 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/ 2023-10-18T17:44:21.368 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 45785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-18T17:44:21.369 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-18T17:44:21.369 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 130553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-18T17:44:21.369 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 68849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-18T17:44:21.369 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ 2023-10-18T17:44:21.369 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-18T17:44:21.369 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 59689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-18T17:44:21.370 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-18T17:44:21.370 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 99113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-18T17:44:21.370 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/ 2023-10-18T17:44:21.370 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-18T17:44:21.370 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/ 2023-10-18T17:44:21.371 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 69201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-18T17:44:21.371 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/ 2023-10-18T17:44:21.371 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 59585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-18T17:44:21.371 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/ 2023-10-18T17:44:21.371 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-18T17:44:21.372 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/ 2023-10-18T17:44:21.372 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-18T17:44:21.372 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-18T17:44:21.372 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 87297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-18T17:44:21.372 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/ 2023-10-18T17:44:21.373 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-18T17:44:21.373 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 97337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-18T17:44:21.373 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-18T17:44:21.373 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 104673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-18T17:44:21.373 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-18T17:44:21.374 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 90257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-18T17:44:21.374 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-18T17:44:21.374 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 81569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-18T17:44:21.374 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 69729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-18T17:44:21.374 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/ 2023-10-18T17:44:21.375 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 78401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-18T17:44:21.375 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 176761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-18T17:44:21.375 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-18T17:44:21.375 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1421497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-18T17:44:21.375 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 128569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-18T17:44:21.375 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-18T17:44:21.376 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 132089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-18T17:44:21.376 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 312081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-18T17:44:21.376 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/ 2023-10-18T17:44:21.376 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-18T17:44:21.376 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 323513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-18T17:44:21.377 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/ 2023-10-18T17:44:21.377 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-18T17:44:21.377 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 193497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-18T17:44:21.377 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-18T17:44:21.377 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 347417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-18T17:44:21.378 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-18T17:44:21.378 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1044433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-18T17:44:21.378 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-18T17:44:21.378 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 166193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-18T17:44:21.378 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-18T17:44:21.379 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-18T17:44:21.379 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/ 2023-10-18T17:44:21.379 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-18T17:44:21.379 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 204345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-18T17:44:21.379 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/ 2023-10-18T17:44:21.379 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-18T17:44:21.380 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 69449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-18T17:44:21.380 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-18T17:44:21.380 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 141225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-18T17:44:21.380 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-18T17:44:21.380 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 62817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-18T17:44:21.381 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31369 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-18T17:44:21.381 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/ 2023-10-18T17:44:21.381 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-18T17:44:21.381 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 62833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-18T17:44:21.381 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 45721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-18T17:44:21.382 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-18T17:44:21.382 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/ 2023-10-18T17:44:21.382 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-18T17:44:21.382 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 281137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-18T17:44:21.382 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-18T17:44:21.383 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-18T17:44:21.383 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-18T17:44:21.383 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 45345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-18T17:44:21.383 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ 2023-10-18T17:44:21.383 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 90465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-18T17:44:21.384 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-18T17:44:21.384 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 295577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-18T17:44:21.384 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-18T17:44:21.384 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 584497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-18T17:44:21.384 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-18T17:44:21.384 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 246209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-18T17:44:21.385 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-18T17:44:21.385 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 925697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-18T17:44:21.385 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-18T17:44:21.385 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 302849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-18T17:44:21.385 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-18T17:44:21.386 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 514681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-18T17:44:21.386 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-18T17:44:21.386 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 93593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-18T17:44:21.386 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-18T17:44:21.386 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 734305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-18T17:44:21.387 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-18T17:44:21.387 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 151769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-18T17:44:21.387 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 86921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-18T17:44:21.387 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/ 2023-10-18T17:44:21.387 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-18T17:44:21.387 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 102289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-18T17:44:21.388 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 122649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-18T17:44:21.388 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/ 2023-10-18T17:44:21.388 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-18T17:44:21.388 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 727601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-18T17:44:21.388 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 278937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-18T17:44:21.389 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-18T17:44:21.389 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-18T17:44:21.389 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1113993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-18T17:44:21.389 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ 2023-10-18T17:44:21.389 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-18T17:44:21.389 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-18T17:44:21.390 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-18T17:44:21.390 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 95665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-18T17:44:21.390 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/ 2023-10-18T17:44:21.390 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-18T17:44:21.390 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 108649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-18T17:44:21.391 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ 2023-10-18T17:44:21.391 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 73441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-18T17:44:21.391 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-18T17:44:21.391 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/ 2023-10-18T17:44:21.391 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 150905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-18T17:44:21.392 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/ 2023-10-18T17:44:21.392 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 136721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-18T17:44:21.392 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/ 2023-10-18T17:44:21.392 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-18T17:44:21.392 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 43145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-18T17:44:21.393 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/ 2023-10-18T17:44:21.393 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-18T17:44:21.393 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 227897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-18T17:44:21.393 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 87081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-18T17:44:21.393 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-18T17:44:21.394 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 672049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-18T17:44:21.394 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/ 2023-10-18T17:44:21.394 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 43017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-18T17:44:21.394 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/ 2023-10-18T17:44:21.394 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 67385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-18T17:44:21.395 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 83985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-18T17:44:21.395 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-18T17:44:21.395 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 189801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-18T17:44:21.395 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/ 2023-10-18T17:44:21.395 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-18T17:44:21.395 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 104241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-18T17:44:21.396 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/ 2023-10-18T17:44:21.396 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1053921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-18T17:44:21.396 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/ 2023-10-18T17:44:21.396 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-18T17:44:21.396 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/ 2023-10-18T17:44:21.397 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-18T17:44:21.397 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 63337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-18T17:44:21.397 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/ 2023-10-18T17:44:21.397 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 61601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-18T17:44:21.397 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 67193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-18T17:44:21.397 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 73409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-18T17:44:21.398 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 68177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-18T17:44:21.398 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/ 2023-10-18T17:44:21.398 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-18T17:44:21.398 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-18T17:44:21.398 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-18T17:44:21.399 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-18T17:44:21.399 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 486961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-18T17:44:21.399 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/ 2023-10-18T17:44:21.399 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 103337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-18T17:44:21.399 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 166161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-18T17:44:21.399 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-18T17:44:21.400 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 78801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-18T17:44:21.400 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/ 2023-10-18T17:44:21.400 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-18T17:44:21.400 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/ 2023-10-18T17:44:21.400 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 86529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-18T17:44:21.400 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/ 2023-10-18T17:44:21.401 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 80273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-18T17:44:21.401 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 83633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-18T17:44:21.401 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/ 2023-10-18T17:44:21.401 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 46241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-18T17:44:21.401 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-18T17:44:21.402 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/ 2023-10-18T17:44:21.402 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-18T17:44:21.402 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/ 2023-10-18T17:44:21.402 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/defxx.ko 2023-10-18T17:44:21.402 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/ 2023-10-18T17:44:21.403 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 216985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-18T17:44:21.403 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 77345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-18T17:44:21.403 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/ 2023-10-18T17:44:21.403 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 235521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-18T17:44:21.403 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/ 2023-10-18T17:44:21.403 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-18T17:44:21.404 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-18T17:44:21.404 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ 2023-10-18T17:44:21.404 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 83769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-18T17:44:21.404 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 62209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-18T17:44:21.404 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-18T17:44:21.404 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/ 2023-10-18T17:44:21.405 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-18T17:44:21.405 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-18T17:44:21.405 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-18T17:44:21.405 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-18T17:44:21.405 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-18T17:44:21.405 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-18T17:44:21.406 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 43953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-18T17:44:21.406 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-18T17:44:21.406 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/ 2023-10-18T17:44:21.406 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-18T17:44:21.406 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-18T17:44:21.407 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ 2023-10-18T17:44:21.407 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-18T17:44:21.407 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-18T17:44:21.407 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-18T17:44:21.407 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-18T17:44:21.408 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-18T17:44:21.408 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-18T17:44:21.408 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-18T17:44:21.408 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-18T17:44:21.408 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-18T17:44:21.408 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-18T17:44:21.409 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 82049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-18T17:44:21.409 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 76177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-18T17:44:21.409 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-18T17:44:21.409 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 97857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-18T17:44:21.409 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-18T17:44:21.410 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-18T17:44:21.410 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-18T17:44:21.410 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-18T17:44:21.410 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-18T17:44:21.410 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/ 2023-10-18T17:44:21.410 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 43569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/plip/plip.ko 2023-10-18T17:44:21.411 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ 2023-10-18T17:44:21.411 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-18T17:44:21.411 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-18T17:44:21.411 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-18T17:44:21.411 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-18T17:44:21.412 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-18T17:44:21.412 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-18T17:44:21.412 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-18T17:44:21.412 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-18T17:44:21.412 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sb1000.ko 2023-10-18T17:44:21.413 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/ 2023-10-18T17:44:21.413 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-18T17:44:21.413 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-18T17:44:21.413 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 64385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-18T17:44:21.413 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/ 2023-10-18T17:44:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 106169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-18T17:44:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-18T17:44:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-18T17:44:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-18T17:44:21.414 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9641 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-18T17:44:21.415 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-18T17:44:21.415 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 87017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-18T17:44:21.415 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/ 2023-10-18T17:44:21.415 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 144297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-18T17:44:21.415 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/ 2023-10-18T17:44:21.415 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 297329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-18T17:44:21.416 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/ 2023-10-18T17:44:21.416 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 144649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-18T17:44:21.416 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ 2023-10-18T17:44:21.416 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-18T17:44:21.416 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/ 2023-10-18T17:44:21.417 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/ 2023-10-18T17:44:21.417 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 264937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-18T17:44:21.417 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 102209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-18T17:44:21.417 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/ 2023-10-18T17:44:21.417 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 133841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-18T17:44:21.418 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-18T17:44:21.418 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 84465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-18T17:44:21.418 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-18T17:44:21.418 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/ 2023-10-18T17:44:21.418 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/ 2023-10-18T17:44:21.419 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-18T17:44:21.419 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-18T17:44:21.419 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-18T17:44:21.419 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pci-stub.ko 2023-10-18T17:44:21.419 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 46529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-18T17:44:21.419 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/ 2023-10-18T17:44:21.420 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-18T17:44:21.420 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 145809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-18T17:44:21.420 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 62137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-18T17:44:21.420 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-18T17:44:21.420 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-18T17:44:21.420 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 112889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-18T17:44:21.421 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/ 2023-10-18T17:44:21.421 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/ 2023-10-18T17:44:21.421 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-18T17:44:21.421 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/ 2023-10-18T17:44:21.421 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-18T17:44:21.421 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/ 2023-10-18T17:44:21.422 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/ 2023-10-18T17:44:21.422 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 70153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-18T17:44:21.422 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/ 2023-10-18T17:44:21.422 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/ 2023-10-18T17:44:21.422 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-18T17:44:21.423 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-18T17:44:21.423 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-18T17:44:21.423 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-18T17:44:21.423 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-18T17:44:21.423 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-18T17:44:21.423 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-18T17:44:21.424 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ 2023-10-18T17:44:21.424 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 67097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-18T17:44:21.424 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-18T17:44:21.424 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-18T17:44:21.424 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 59665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-18T17:44:21.425 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-18T17:44:21.425 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 127569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-18T17:44:21.425 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-18T17:44:21.425 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-18T17:44:21.425 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-18T17:44:21.425 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-18T17:44:21.426 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-18T17:44:21.426 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-18T17:44:21.426 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-18T17:44:21.426 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/ 2023-10-18T17:44:21.426 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-18T17:44:21.427 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-18T17:44:21.427 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 43497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-18T17:44:21.427 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-18T17:44:21.427 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10377 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-18T17:44:21.427 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-18T17:44:21.427 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-18T17:44:21.428 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-18T17:44:21.428 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-18T17:44:21.428 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-18T17:44:21.428 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-18T17:44:21.428 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-18T17:44:21.428 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 57569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-18T17:44:21.429 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/ 2023-10-18T17:44:21.429 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ 2023-10-18T17:44:21.429 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-18T17:44:21.429 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-18T17:44:21.429 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 55057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-18T17:44:21.429 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-18T17:44:21.430 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-18T17:44:21.430 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-18T17:44:21.430 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-18T17:44:21.430 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-18T17:44:21.430 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29281 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-18T17:44:21.431 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23217 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-18T17:44:21.431 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-18T17:44:21.431 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-18T17:44:21.431 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-18T17:44:21.431 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-18T17:44:21.432 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-18T17:44:21.432 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-18T17:44:21.432 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-18T17:44:21.432 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-18T17:44:21.432 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-18T17:44:21.433 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/max8925_power.ko 2023-10-18T17:44:21.433 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-18T17:44:21.433 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-18T17:44:21.433 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/test_power.ko 2023-10-18T17:44:21.433 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-18T17:44:21.433 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-18T17:44:21.434 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-18T17:44:21.434 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-18T17:44:21.434 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/ 2023-10-18T17:44:21.434 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-18T17:44:21.434 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-18T17:44:21.435 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/ 2023-10-18T17:44:21.435 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/ 2023-10-18T17:44:21.435 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-18T17:44:21.435 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-18T17:44:21.435 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-18T17:44:21.436 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-18T17:44:21.436 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ 2023-10-18T17:44:21.436 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-18T17:44:21.436 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-18T17:44:21.436 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/ 2023-10-18T17:44:21.436 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-18T17:44:21.437 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-18T17:44:21.437 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-18T17:44:21.437 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-18T17:44:21.437 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-18T17:44:21.437 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-18T17:44:21.438 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-18T17:44:21.438 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 68577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-18T17:44:21.438 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-18T17:44:21.438 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18217 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-18T17:44:21.438 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-18T17:44:21.439 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-18T17:44:21.439 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13369 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-18T17:44:21.439 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-18T17:44:21.439 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23705 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-18T17:44:21.439 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-18T17:44:21.439 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-18T17:44:21.440 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-18T17:44:21.440 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-18T17:44:21.440 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-18T17:44:21.440 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-18T17:44:21.440 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-18T17:44:21.441 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-18T17:44:21.441 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-18T17:44:21.441 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-18T17:44:21.441 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-18T17:44:21.441 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-18T17:44:21.442 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-18T17:44:21.442 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-18T17:44:21.442 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-18T17:44:21.442 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-18T17:44:21.442 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-18T17:44:21.443 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 46633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-18T17:44:21.443 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-18T17:44:21.443 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-18T17:44:21.443 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-18T17:44:21.443 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-18T17:44:21.443 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-18T17:44:21.444 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-18T17:44:21.444 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-18T17:44:21.444 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-18T17:44:21.444 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-18T17:44:21.444 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-18T17:44:21.445 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-18T17:44:21.445 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-18T17:44:21.445 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-18T17:44:21.445 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-18T17:44:21.445 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-18T17:44:21.445 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-18T17:44:21.446 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-18T17:44:21.446 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-18T17:44:21.446 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ 2023-10-18T17:44:21.446 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 77617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-18T17:44:21.446 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 57665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-18T17:44:21.447 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 55169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-18T17:44:21.447 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 135185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-18T17:44:21.447 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-18T17:44:21.447 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/ 2023-10-18T17:44:21.447 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 272305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-18T17:44:21.448 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 100617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-18T17:44:21.448 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/ 2023-10-18T17:44:21.448 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 356265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-18T17:44:21.448 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 295801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-18T17:44:21.448 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/ 2023-10-18T17:44:21.449 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 189185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-18T17:44:21.449 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-18T17:44:21.449 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/ 2023-10-18T17:44:21.449 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 106881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-18T17:44:21.449 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-18T17:44:21.449 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/ 2023-10-18T17:44:21.450 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 246985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-18T17:44:21.450 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/ 2023-10-18T17:44:21.450 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1171945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-18T17:44:21.450 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/ 2023-10-18T17:44:21.450 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 237273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-18T17:44:21.451 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/ 2023-10-18T17:44:21.451 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 115369 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-18T17:44:21.451 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-18T17:44:21.451 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/ 2023-10-18T17:44:21.451 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 308921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-18T17:44:21.451 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/ 2023-10-18T17:44:21.452 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-18T17:44:21.452 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 76145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-18T17:44:21.452 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-18T17:44:21.452 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 101449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-18T17:44:21.452 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 146689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-18T17:44:21.453 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 75553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-18T17:44:21.453 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-18T17:44:21.453 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/ 2023-10-18T17:44:21.453 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 212425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-18T17:44:21.453 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-18T17:44:21.454 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/ 2023-10-18T17:44:21.454 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 78209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-18T17:44:21.454 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 158505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-18T17:44:21.454 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fdomain.ko 2023-10-18T17:44:21.454 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/ 2023-10-18T17:44:21.455 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 263489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-18T17:44:21.455 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 201817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-18T17:44:21.455 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-18T17:44:21.455 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-18T17:44:21.455 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/imm.ko 2023-10-18T17:44:21.455 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-18T17:44:21.456 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 266537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-18T17:44:21.456 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 101881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-18T17:44:21.456 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/ 2023-10-18T17:44:21.456 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 317153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-18T17:44:21.456 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-18T17:44:21.457 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 52961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-18T17:44:21.457 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/ 2023-10-18T17:44:21.457 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 314865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-18T17:44:21.457 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 145585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-18T17:44:21.457 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-18T17:44:21.458 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/ 2023-10-18T17:44:21.458 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 216665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-18T17:44:21.458 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/ 2023-10-18T17:44:21.458 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 2197249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-18T17:44:21.458 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/ 2023-10-18T17:44:21.459 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 73569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-18T17:44:21.459 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-18T17:44:21.459 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 310617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-18T17:44:21.459 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 84721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-18T17:44:21.459 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/ 2023-10-18T17:44:21.459 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 659777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-18T17:44:21.460 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/ 2023-10-18T17:44:21.460 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 119561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-18T17:44:21.460 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 78129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-18T17:44:21.460 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/ 2023-10-18T17:44:21.460 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 81625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-18T17:44:21.461 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-18T17:44:21.461 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-18T17:44:21.461 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-18T17:44:21.461 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/ 2023-10-18T17:44:21.461 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 364105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-18T17:44:21.462 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 123617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-18T17:44:21.462 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ppa.ko 2023-10-18T17:44:21.462 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 55361 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-18T17:44:21.462 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/ 2023-10-18T17:44:21.462 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1876625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-18T17:44:21.462 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 85289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-18T17:44:21.463 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/ 2023-10-18T17:44:21.463 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 559033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-18T17:44:21.463 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-18T17:44:21.463 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-18T17:44:21.463 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 238825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-18T17:44:21.464 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 173937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-18T17:44:21.464 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 279313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-18T17:44:21.464 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 97729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-18T17:44:21.464 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 73881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-18T17:44:21.464 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-18T17:44:21.464 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-18T17:44:21.465 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 115889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-18T17:44:21.465 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 46825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-18T17:44:21.465 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-18T17:44:21.465 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 162121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-18T17:44:21.465 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-18T17:44:21.465 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-18T17:44:21.466 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-18T17:44:21.466 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-18T17:44:21.466 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ 2023-10-18T17:44:21.466 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 129441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-18T17:44:21.466 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/ 2023-10-18T17:44:21.467 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/ 2023-10-18T17:44:21.467 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 736601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-18T17:44:21.467 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/ 2023-10-18T17:44:21.467 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-18T17:44:21.467 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-18T17:44:21.467 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 43025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-18T17:44:21.468 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 930321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-18T17:44:21.468 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-18T17:44:21.468 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 122521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-18T17:44:21.468 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/ 2023-10-18T17:44:21.468 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 75593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-18T17:44:21.468 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/ 2023-10-18T17:44:21.469 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/ 2023-10-18T17:44:21.469 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-18T17:44:21.469 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-18T17:44:21.469 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-18T17:44:21.469 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-18T17:44:21.470 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-18T17:44:21.470 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-18T17:44:21.470 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-18T17:44:21.470 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-18T17:44:21.470 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20281 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-18T17:44:21.471 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-18T17:44:21.471 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-18T17:44:21.471 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-18T17:44:21.471 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-18T17:44:21.471 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-18T17:44:21.471 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-18T17:44:21.472 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-18T17:44:21.472 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-18T17:44:21.472 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ 2023-10-18T17:44:21.472 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ 2023-10-18T17:44:21.472 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 78425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-18T17:44:21.473 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 57865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/moxa.ko 2023-10-18T17:44:21.473 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 50953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/mxser.ko 2023-10-18T17:44:21.473 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 108601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-18T17:44:21.473 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-18T17:44:21.473 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 67929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-18T17:44:21.474 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/ 2023-10-18T17:44:21.474 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/ 2023-10-18T17:44:21.474 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-18T17:44:21.474 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 59497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-18T17:44:21.474 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-18T17:44:21.475 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-18T17:44:21.475 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-18T17:44:21.475 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 72425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-18T17:44:21.475 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/ 2023-10-18T17:44:21.475 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 103073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-18T17:44:21.476 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/rp2.ko 2023-10-18T17:44:21.476 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 52817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-18T17:44:21.476 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/ 2023-10-18T17:44:21.476 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ 2023-10-18T17:44:21.476 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 531265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-18T17:44:21.476 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ 2023-10-18T17:44:21.477 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-18T17:44:21.477 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-18T17:44:21.477 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/ 2023-10-18T17:44:21.477 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-18T17:44:21.477 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-18T17:44:21.478 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11281 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-18T17:44:21.478 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-18T17:44:21.478 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_mf624.ko 2023-10-18T17:44:21.478 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_netx.ko 2023-10-18T17:44:21.479 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-18T17:44:21.479 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-18T17:44:21.479 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-18T17:44:21.479 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/ 2023-10-18T17:44:21.479 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/ 2023-10-18T17:44:21.479 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 73297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-18T17:44:21.480 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ 2023-10-18T17:44:21.480 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 144377 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-18T17:44:21.480 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-18T17:44:21.480 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-18T17:44:21.480 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-18T17:44:21.481 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/ 2023-10-18T17:44:21.481 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 87089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-18T17:44:21.481 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-18T17:44:21.481 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47369 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-18T17:44:21.481 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 89857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-18T17:44:21.481 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ 2023-10-18T17:44:21.482 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/common/ulpi.ko 2023-10-18T17:44:21.482 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/ 2023-10-18T17:44:21.482 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-18T17:44:21.482 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/ 2023-10-18T17:44:21.482 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-18T17:44:21.483 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-18T17:44:21.483 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 444553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-18T17:44:21.483 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/ 2023-10-18T17:44:21.483 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/ 2023-10-18T17:44:21.483 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-18T17:44:21.484 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-18T17:44:21.484 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-18T17:44:21.484 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-18T17:44:21.484 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-18T17:44:21.484 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-18T17:44:21.484 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 92489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-18T17:44:21.485 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-18T17:44:21.485 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 122081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-18T17:44:21.485 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 57617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-18T17:44:21.485 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-18T17:44:21.485 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-18T17:44:21.486 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 57729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-18T17:44:21.486 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 85609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-18T17:44:21.486 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-18T17:44:21.486 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 58113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-18T17:44:21.486 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 67337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-18T17:44:21.487 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/ 2023-10-18T17:44:21.487 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-18T17:44:21.487 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-18T17:44:21.487 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-18T17:44:21.487 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-18T17:44:21.487 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-18T17:44:21.488 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-18T17:44:21.488 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-18T17:44:21.488 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-18T17:44:21.488 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-18T17:44:21.488 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-18T17:44:21.489 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-18T17:44:21.489 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-18T17:44:21.489 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 62185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-18T17:44:21.489 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-18T17:44:21.489 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 161225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-18T17:44:21.490 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/ 2023-10-18T17:44:21.490 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17705 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-18T17:44:21.490 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-18T17:44:21.490 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 157681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-18T17:44:21.490 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-18T17:44:21.491 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 52721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-18T17:44:21.491 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 69705 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-18T17:44:21.491 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 56033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-18T17:44:21.491 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 76057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-18T17:44:21.491 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 75073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-18T17:44:21.492 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 73169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-18T17:44:21.492 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 65809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-18T17:44:21.492 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-18T17:44:21.492 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 81505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-18T17:44:21.492 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 156073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-18T17:44:21.492 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ 2023-10-18T17:44:21.493 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-18T17:44:21.493 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 80113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-18T17:44:21.493 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 87353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-18T17:44:21.493 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 78873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-18T17:44:21.493 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-18T17:44:21.494 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-18T17:44:21.494 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-18T17:44:21.494 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-18T17:44:21.494 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/ 2023-10-18T17:44:21.494 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-18T17:44:21.494 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-18T17:44:21.495 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ 2023-10-18T17:44:21.495 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-18T17:44:21.495 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-18T17:44:21.495 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-18T17:44:21.495 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/cytherm.ko 2023-10-18T17:44:21.496 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ehset.ko 2023-10-18T17:44:21.496 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-18T17:44:21.496 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-18T17:44:21.496 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-18T17:44:21.496 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-18T17:44:21.497 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-18T17:44:21.497 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-18T17:44:21.497 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-18T17:44:21.497 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-18T17:44:21.497 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/lvstest.ko 2023-10-18T17:44:21.498 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-18T17:44:21.498 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 64841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-18T17:44:21.498 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-18T17:44:21.498 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-18T17:44:21.498 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-18T17:44:21.498 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-18T17:44:21.499 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 70873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbtest.ko 2023-10-18T17:44:21.499 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-18T17:44:21.499 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-18T17:44:21.499 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/ 2023-10-18T17:44:21.499 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 72953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/mon/usbmon.ko 2023-10-18T17:44:21.500 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/ 2023-10-18T17:44:21.500 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 301545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-18T17:44:21.500 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/ 2023-10-18T17:44:21.500 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-18T17:44:21.500 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-18T17:44:21.500 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-18T17:44:21.501 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-18T17:44:21.501 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ 2023-10-18T17:44:21.501 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-18T17:44:21.501 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-18T17:44:21.501 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-18T17:44:21.501 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-18T17:44:21.502 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 70257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-18T17:44:21.502 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-18T17:44:21.502 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 58217 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-18T17:44:21.502 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-18T17:44:21.502 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-18T17:44:21.503 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/f81232.ko 2023-10-18T17:44:21.503 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 175137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-18T17:44:21.503 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-18T17:44:21.503 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 137081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-18T17:44:21.503 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 124833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-18T17:44:21.504 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 67769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-18T17:44:21.504 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-18T17:44:21.504 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-18T17:44:21.504 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 58233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-18T17:44:21.504 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 107521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-18T17:44:21.504 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-18T17:44:21.505 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-18T17:44:21.505 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-18T17:44:21.505 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-18T17:44:21.505 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-18T17:44:21.505 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-18T17:44:21.506 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 78217 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-18T17:44:21.506 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 56337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mxuport.ko 2023-10-18T17:44:21.506 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-18T17:44:21.506 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-18T17:44:21.506 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-18T17:44:21.506 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 190633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-18T17:44:21.507 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-18T17:44:21.507 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-18T17:44:21.507 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-18T17:44:21.507 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-18T17:44:21.507 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-18T17:44:21.507 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-18T17:44:21.508 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 51625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-18T17:44:21.508 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-18T17:44:21.508 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-18T17:44:21.508 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-18T17:44:21.508 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 61489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-18T17:44:21.509 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-18T17:44:21.509 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-18T17:44:21.509 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-18T17:44:21.509 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 125609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usbserial.ko 2023-10-18T17:44:21.509 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-18T17:44:21.509 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-18T17:44:21.510 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-18T17:44:21.510 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-18T17:44:21.510 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ 2023-10-18T17:44:21.510 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 52913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-18T17:44:21.510 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-18T17:44:21.510 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-18T17:44:21.511 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-18T17:44:21.511 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-18T17:44:21.511 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-18T17:44:21.511 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-18T17:44:21.511 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19361 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-18T17:44:21.512 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-18T17:44:21.512 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-18T17:44:21.512 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-18T17:44:21.512 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-18T17:44:21.512 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-18T17:44:21.513 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-18T17:44:21.513 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 179969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-18T17:44:21.513 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/ 2023-10-18T17:44:21.513 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 67481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-18T17:44:21.513 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 72177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-18T17:44:21.514 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 107001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-18T17:44:21.514 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/ 2023-10-18T17:44:21.514 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/ 2023-10-18T17:44:21.514 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 154297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-18T17:44:21.514 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-18T17:44:21.515 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 108737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-18T17:44:21.515 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 77609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-18T17:44:21.515 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/ 2023-10-18T17:44:21.515 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 113873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-18T17:44:21.515 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-18T17:44:21.516 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 58089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-18T17:44:21.516 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 89249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-18T17:44:21.516 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/ 2023-10-18T17:44:21.516 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/ 2023-10-18T17:44:21.516 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-18T17:44:21.517 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-18T17:44:21.517 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-18T17:44:21.517 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-18T17:44:21.517 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-18T17:44:21.517 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-18T17:44:21.518 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-18T17:44:21.518 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/bd6107.ko 2023-10-18T17:44:21.518 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-18T17:44:21.518 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-18T17:44:21.518 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-18T17:44:21.519 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-18T17:44:21.519 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-18T17:44:21.519 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-18T17:44:21.519 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-18T17:44:21.519 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-18T17:44:21.520 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-18T17:44:21.520 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-18T17:44:21.520 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-18T17:44:21.520 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-18T17:44:21.520 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-18T17:44:21.521 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/ 2023-10-18T17:44:21.521 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ 2023-10-18T17:44:21.521 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-18T17:44:21.521 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-18T17:44:21.521 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-18T17:44:21.521 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-18T17:44:21.522 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/ 2023-10-18T17:44:21.522 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-18T17:44:21.522 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-18T17:44:21.522 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-18T17:44:21.522 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-18T17:44:21.523 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-18T17:44:21.523 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-18T17:44:21.523 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-18T17:44:21.523 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-18T17:44:21.523 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-18T17:44:21.524 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-18T17:44:21.524 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 56425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-18T17:44:21.524 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 102673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-18T17:44:21.524 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ 2023-10-18T17:44:21.524 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-18T17:44:21.524 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-18T17:44:21.525 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-18T17:44:21.525 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-18T17:44:21.525 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-18T17:44:21.525 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-18T17:44:21.525 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-18T17:44:21.526 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-18T17:44:21.526 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-18T17:44:21.526 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-18T17:44:21.526 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-18T17:44:21.526 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-18T17:44:21.527 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-18T17:44:21.527 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-18T17:44:21.527 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-18T17:44:21.527 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-18T17:44:21.527 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-18T17:44:21.527 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-18T17:44:21.528 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-18T17:44:21.528 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-18T17:44:21.528 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-18T17:44:21.528 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-18T17:44:21.528 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-18T17:44:21.529 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-18T17:44:21.529 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-18T17:44:21.529 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-18T17:44:21.529 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22265 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-18T17:44:21.529 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-18T17:44:21.530 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-18T17:44:21.530 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-18T17:44:21.530 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-18T17:44:21.530 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-18T17:44:21.530 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-18T17:44:21.531 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-18T17:44:21.531 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-18T17:44:21.531 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-18T17:44:21.531 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-18T17:44:21.531 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-18T17:44:21.531 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-18T17:44:21.532 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-18T17:44:21.532 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-18T17:44:21.532 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-18T17:44:21.532 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-18T17:44:21.532 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-18T17:44:21.533 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-18T17:44:21.533 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-18T17:44:21.533 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-18T17:44:21.533 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-18T17:44:21.533 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/ 2023-10-18T17:44:21.533 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-18T17:44:21.534 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-18T17:44:21.534 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-18T17:44:21.534 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/ 2023-10-18T17:44:21.534 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 176441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-18T17:44:21.534 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-18T17:44:21.534 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 71249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-scsiback.ko 2023-10-18T17:44:21.535 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/ 2023-10-18T17:44:21.535 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-18T17:44:21.535 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ 2023-10-18T17:44:21.535 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/ 2023-10-18T17:44:21.535 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 166425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-18T17:44:21.536 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/ 2023-10-18T17:44:21.536 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 61073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/adfs/adfs.ko 2023-10-18T17:44:21.536 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/ 2023-10-18T17:44:21.536 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 163593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-18T17:44:21.536 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/ 2023-10-18T17:44:21.536 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 932817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/afs/kafs.ko 2023-10-18T17:44:21.537 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/ 2023-10-18T17:44:21.537 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 101057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/autofs/autofs4.ko 2023-10-18T17:44:21.537 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/ 2023-10-18T17:44:21.537 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 64545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-18T17:44:21.537 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/ 2023-10-18T17:44:21.537 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/bfs/bfs.ko 2023-10-18T17:44:21.538 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-18T17:44:21.538 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/ 2023-10-18T17:44:21.538 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 3402585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-18T17:44:21.538 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/ 2023-10-18T17:44:21.538 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 306073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-18T17:44:21.539 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ 2023-10-18T17:44:21.539 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1323873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-18T17:44:21.539 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/ 2023-10-18T17:44:21.539 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 100801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-18T17:44:21.539 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/ 2023-10-18T17:44:21.539 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 108393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/configfs/configfs.ko 2023-10-18T17:44:21.540 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/ 2023-10-18T17:44:21.540 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 29625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-18T17:44:21.540 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/ 2023-10-18T17:44:21.540 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 642577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-18T17:44:21.540 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/ 2023-10-18T17:44:21.540 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/efs/efs.ko 2023-10-18T17:44:21.541 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/ 2023-10-18T17:44:21.541 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1505025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-18T17:44:21.541 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/ 2023-10-18T17:44:21.541 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-18T17:44:21.541 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/ 2023-10-18T17:44:21.542 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/freevxfs/freevxfs.ko 2023-10-18T17:44:21.542 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/ 2023-10-18T17:44:21.542 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 226481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-18T17:44:21.542 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/ 2023-10-18T17:44:21.542 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-18T17:44:21.543 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/ 2023-10-18T17:44:21.543 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1231529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-18T17:44:21.543 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/ 2023-10-18T17:44:21.543 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 141209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-18T17:44:21.543 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/ 2023-10-18T17:44:21.543 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 222817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-18T17:44:21.544 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/ 2023-10-18T17:44:21.544 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 192329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hpfs/hpfs.ko 2023-10-18T17:44:21.544 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/ 2023-10-18T17:44:21.544 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 74065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-18T17:44:21.544 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/ 2023-10-18T17:44:21.545 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 419497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-18T17:44:21.545 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/ 2023-10-18T17:44:21.545 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 280313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-18T17:44:21.545 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/ 2023-10-18T17:44:21.545 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 89193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-18T17:44:21.545 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/ 2023-10-18T17:44:21.546 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 124649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-18T17:44:21.546 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/ 2023-10-18T17:44:21.546 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/ 2023-10-18T17:44:21.546 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 81449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-18T17:44:21.546 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/ 2023-10-18T17:44:21.547 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 59449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-18T17:44:21.547 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/ 2023-10-18T17:44:21.547 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 117513 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-18T17:44:21.547 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1055649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-18T17:44:21.547 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv2.ko 2023-10-18T17:44:21.547 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 120177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-18T17:44:21.548 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 2041689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-18T17:44:21.548 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/ 2023-10-18T17:44:21.548 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-18T17:44:21.548 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-18T17:44:21.548 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/ 2023-10-18T17:44:21.548 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1516809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-18T17:44:21.548 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/ 2023-10-18T17:44:21.549 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 592833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-18T17:44:21.549 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/ 2023-10-18T17:44:21.549 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-18T17:44:21.549 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-18T17:44:21.549 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-18T17:44:21.550 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-18T17:44:21.550 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-18T17:44:21.550 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-18T17:44:21.550 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-18T17:44:21.550 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-18T17:44:21.550 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-18T17:44:21.551 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-18T17:44:21.551 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-18T17:44:21.551 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ascii.ko 2023-10-18T17:44:21.551 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-18T17:44:21.551 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-18T17:44:21.552 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-18T17:44:21.552 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-18T17:44:21.552 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-18T17:44:21.552 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-18T17:44:21.552 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-18T17:44:21.552 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-18T17:44:21.553 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-18T17:44:21.553 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-18T17:44:21.553 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-18T17:44:21.553 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-18T17:44:21.553 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-18T17:44:21.553 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-18T17:44:21.554 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-18T17:44:21.554 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-18T17:44:21.554 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-18T17:44:21.554 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-18T17:44:21.554 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 94137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-18T17:44:21.554 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 142673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-18T17:44:21.555 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 165745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-18T17:44:21.555 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 118121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-18T17:44:21.555 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-18T17:44:21.555 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-18T17:44:21.555 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-18T17:44:21.556 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-18T17:44:21.557 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-18T17:44:21.557 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-18T17:44:21.557 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-18T17:44:21.558 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-18T17:44:21.558 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-18T17:44:21.558 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-18T17:44:21.558 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-18T17:44:21.558 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-18T17:44:21.559 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-18T17:44:21.559 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-18T17:44:21.559 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-18T17:44:21.559 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-18T17:44:21.559 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-18T17:44:21.560 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ 2023-10-18T17:44:21.560 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 219417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ntfs/ntfs.ko 2023-10-18T17:44:21.560 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ 2023-10-18T17:44:21.560 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ 2023-10-18T17:44:21.560 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 244409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-18T17:44:21.560 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ 2023-10-18T17:44:21.561 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 440737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-18T17:44:21.561 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ 2023-10-18T17:44:21.561 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-18T17:44:21.561 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 2592433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-18T17:44:21.561 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-18T17:44:21.562 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-18T17:44:21.562 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-18T17:44:21.562 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/ 2023-10-18T17:44:21.562 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/omfs/omfs.ko 2023-10-18T17:44:21.562 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/ 2023-10-18T17:44:21.563 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 352153 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-18T17:44:21.563 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ 2023-10-18T17:44:21.563 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-18T17:44:21.563 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/ 2023-10-18T17:44:21.563 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx4/qnx4.ko 2023-10-18T17:44:21.563 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/ 2023-10-18T17:44:21.564 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 46689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/qnx6/qnx6.ko 2023-10-18T17:44:21.564 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/ 2023-10-18T17:44:21.564 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_tree.ko 2023-10-18T17:44:21.564 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14361 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v1.ko 2023-10-18T17:44:21.564 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/quota/quota_v2.ko 2023-10-18T17:44:21.565 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/ 2023-10-18T17:44:21.565 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 532673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-18T17:44:21.565 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/ 2023-10-18T17:44:21.565 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28641 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-18T17:44:21.565 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/ 2023-10-18T17:44:21.565 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/ 2023-10-18T17:44:21.566 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 2668121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-18T17:44:21.566 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/ 2023-10-18T17:44:21.566 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-18T17:44:21.566 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-18T17:44:21.566 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/ 2023-10-18T17:44:21.567 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 134641 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-18T17:44:21.567 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/ 2023-10-18T17:44:21.567 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 90721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-18T17:44:21.567 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/ 2023-10-18T17:44:21.567 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 247737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-18T17:44:21.568 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ 2023-10-18T17:44:21.568 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 162497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-18T17:44:21.568 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/ 2023-10-18T17:44:21.568 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 4010729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-18T17:44:21.568 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/ 2023-10-18T17:44:21.569 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/ 2023-10-18T17:44:21.569 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 75489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/locking/locktorture.ko 2023-10-18T17:44:21.569 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/ 2023-10-18T17:44:21.569 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/time/test_udelay.ko 2023-10-18T17:44:21.569 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-18T17:44:21.569 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ 2023-10-18T17:44:21.570 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 6833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-18T17:44:21.570 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 6529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc7.ko 2023-10-18T17:44:21.570 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-18T17:44:21.570 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/ 2023-10-18T17:44:21.570 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-18T17:44:21.571 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-18T17:44:21.571 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/interval_tree_test.ko 2023-10-18T17:44:21.571 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-18T17:44:21.571 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-18T17:44:21.571 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/ 2023-10-18T17:44:21.572 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-18T17:44:21.572 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-18T17:44:21.572 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/ 2023-10-18T17:44:21.572 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 6617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-18T17:44:21.572 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/memory-notifier-error-inject.ko 2023-10-18T17:44:21.572 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/notifier-error-inject.ko 2023-10-18T17:44:21.573 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/percpu_test.ko 2023-10-18T17:44:21.573 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/pm-notifier-error-inject.ko 2023-10-18T17:44:21.573 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/ 2023-10-18T17:44:21.573 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 143433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-18T17:44:21.573 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/rbtree_test.ko 2023-10-18T17:44:21.574 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/ 2023-10-18T17:44:21.574 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-18T17:44:21.574 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-kstrtox.ko 2023-10-18T17:44:21.574 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test-string_helpers.ko 2023-10-18T17:44:21.574 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 4760425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_bpf.ko 2023-10-18T17:44:21.575 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_firmware.ko 2023-10-18T17:44:21.575 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 6369 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_module.ko 2023-10-18T17:44:21.575 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/test_user_copy.ko 2023-10-18T17:44:21.575 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-18T17:44:21.575 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-18T17:44:21.575 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-18T17:44:21.576 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/ 2023-10-18T17:44:21.576 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/xz/xz_dec_test.ko 2023-10-18T17:44:21.576 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/ 2023-10-18T17:44:21.576 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-18T17:44:21.576 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ 2023-10-18T17:44:21.577 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/ 2023-10-18T17:44:21.577 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 55737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-18T17:44:21.577 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-18T17:44:21.577 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-18T17:44:21.577 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-18T17:44:21.577 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 6985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-18T17:44:21.578 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-18T17:44:21.578 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-18T17:44:21.578 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-18T17:44:21.578 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/ 2023-10-18T17:44:21.578 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-18T17:44:21.579 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-18T17:44:21.579 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-18T17:44:21.579 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-18T17:44:21.579 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-18T17:44:21.579 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/ 2023-10-18T17:44:21.580 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 81737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-18T17:44:21.580 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/ 2023-10-18T17:44:21.580 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 198337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-18T17:44:21.580 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-18T17:44:21.580 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26345 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-18T17:44:21.581 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-18T17:44:21.581 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/ 2023-10-18T17:44:21.581 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 649297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-18T17:44:21.581 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/ 2023-10-18T17:44:21.581 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 63809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-18T17:44:21.581 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 777689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-18T17:44:21.582 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ 2023-10-18T17:44:21.582 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-18T17:44:21.582 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-18T17:44:21.582 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-18T17:44:21.582 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-18T17:44:21.583 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-18T17:44:21.583 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-18T17:44:21.583 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-18T17:44:21.583 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-18T17:44:21.583 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13217 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-18T17:44:21.583 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-18T17:44:21.584 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-18T17:44:21.584 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8385 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-18T17:44:21.584 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-18T17:44:21.584 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-18T17:44:21.584 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-18T17:44:21.585 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-18T17:44:21.585 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-18T17:44:21.585 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-18T17:44:21.585 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-18T17:44:21.585 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-18T17:44:21.585 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 55929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-18T17:44:21.586 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-18T17:44:21.586 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-18T17:44:21.586 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/ 2023-10-18T17:44:21.586 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1047017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-18T17:44:21.586 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/ 2023-10-18T17:44:21.587 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 125697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-18T17:44:21.587 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/ 2023-10-18T17:44:21.587 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 194209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-18T17:44:21.587 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-18T17:44:21.587 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 57881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-18T17:44:21.588 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-18T17:44:21.588 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/ 2023-10-18T17:44:21.588 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 390473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-18T17:44:21.588 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-18T17:44:21.588 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-18T17:44:21.588 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-18T17:44:21.589 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-18T17:44:21.589 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/ 2023-10-18T17:44:21.589 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 99729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/hsr/hsr.ko 2023-10-18T17:44:21.589 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ 2023-10-18T17:44:21.589 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ 2023-10-18T17:44:21.590 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 45233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-18T17:44:21.590 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 254625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-18T17:44:21.590 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-18T17:44:21.590 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ 2023-10-18T17:44:21.590 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-18T17:44:21.590 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-18T17:44:21.591 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 48401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-18T17:44:21.591 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-18T17:44:21.591 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 48657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-18T17:44:21.591 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-18T17:44:21.591 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 61225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-18T17:44:21.592 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-18T17:44:21.592 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-18T17:44:21.592 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-18T17:44:21.592 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ 2023-10-18T17:44:21.592 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37641 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-18T17:44:21.593 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-18T17:44:21.593 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-18T17:44:21.593 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-18T17:44:21.593 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-18T17:44:21.593 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-18T17:44:21.593 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-18T17:44:21.594 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-18T17:44:21.594 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-18T17:44:21.594 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11305 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-18T17:44:21.594 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-18T17:44:21.594 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-18T17:44:21.595 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-18T17:44:21.595 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-18T17:44:21.595 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-18T17:44:21.595 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-18T17:44:21.595 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-18T17:44:21.596 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18225 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-18T17:44:21.596 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-18T17:44:21.596 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-18T17:44:21.596 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-18T17:44:21.596 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-18T17:44:21.597 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-18T17:44:21.597 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-18T17:44:21.597 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-18T17:44:21.597 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-18T17:44:21.597 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-18T17:44:21.597 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15369 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-18T17:44:21.598 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-18T17:44:21.598 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-18T17:44:21.598 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-18T17:44:21.598 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-18T17:44:21.598 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-18T17:44:21.599 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-18T17:44:21.599 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-18T17:44:21.599 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-18T17:44:21.599 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-18T17:44:21.599 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-18T17:44:21.600 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-18T17:44:21.600 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-18T17:44:21.600 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 41665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-18T17:44:21.600 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-18T17:44:21.600 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ 2023-10-18T17:44:21.600 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-18T17:44:21.601 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-18T17:44:21.601 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-18T17:44:21.601 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 80897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_gre.ko 2023-10-18T17:44:21.601 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 93577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-18T17:44:21.601 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-18T17:44:21.601 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-18T17:44:21.602 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-18T17:44:21.602 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19017 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/mip6.ko 2023-10-18T17:44:21.602 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ 2023-10-18T17:44:21.602 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-18T17:44:21.602 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-18T17:44:21.603 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-18T17:44:21.603 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-18T17:44:21.603 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-18T17:44:21.603 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-18T17:44:21.603 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-18T17:44:21.603 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-18T17:44:21.604 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-18T17:44:21.604 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-18T17:44:21.604 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-18T17:44:21.604 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-18T17:44:21.604 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-18T17:44:21.604 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11121 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-18T17:44:21.605 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-18T17:44:21.605 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-18T17:44:21.605 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-18T17:44:21.605 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-18T17:44:21.605 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-18T17:44:21.605 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25945 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-18T17:44:21.606 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-18T17:44:21.606 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-18T17:44:21.606 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-18T17:44:21.606 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-18T17:44:21.606 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9009 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-18T17:44:21.607 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 74073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-18T17:44:21.607 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-18T17:44:21.607 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-18T17:44:21.607 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/ 2023-10-18T17:44:21.607 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 82833 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-18T17:44:21.607 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/ 2023-10-18T17:44:21.608 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 125281 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-18T17:44:21.608 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-18T17:44:21.608 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 25953 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-18T17:44:21.608 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-18T17:44:21.608 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36713 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-18T17:44:21.608 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-18T17:44:21.609 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 53241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-18T17:44:21.609 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/ 2023-10-18T17:44:21.609 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 47977 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/lapb/lapb.ko 2023-10-18T17:44:21.609 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/ 2023-10-18T17:44:21.609 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-18T17:44:21.609 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 267777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc2.ko 2023-10-18T17:44:21.610 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/ 2023-10-18T17:44:21.610 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 290417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-18T17:44:21.610 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/ 2023-10-18T17:44:21.610 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-18T17:44:21.610 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ 2023-10-18T17:44:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ 2023-10-18T17:44:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 119585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-18T17:44:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-18T17:44:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26913 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-18T17:44:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26369 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-18T17:44:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 100297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-18T17:44:21.612 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 101441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-18T17:44:21.612 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 102817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-18T17:44:21.612 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 103481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-18T17:44:21.612 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 112849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-18T17:44:21.612 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 60329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-18T17:44:21.613 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 109593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-18T17:44:21.613 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 113129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-18T17:44:21.613 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 112585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-18T17:44:21.613 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 111585 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-18T17:44:21.613 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 114465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-18T17:44:21.613 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-18T17:44:21.614 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ 2023-10-18T17:44:21.614 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 411729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-18T17:44:21.614 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-18T17:44:21.614 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-18T17:44:21.614 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-18T17:44:21.615 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-18T17:44:21.615 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-18T17:44:21.615 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-18T17:44:21.615 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7689 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-18T17:44:21.615 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10681 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-18T17:44:21.615 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-18T17:44:21.616 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-18T17:44:21.616 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-18T17:44:21.616 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-18T17:44:21.616 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10993 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-18T17:44:21.616 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-18T17:44:21.616 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 408257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-18T17:44:21.617 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-18T17:44:21.617 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-18T17:44:21.617 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33761 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-18T17:44:21.617 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 118145 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-18T17:44:21.617 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-18T17:44:21.617 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-18T17:44:21.618 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 107297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-18T17:44:21.618 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-18T17:44:21.618 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-18T17:44:21.618 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66641 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-18T17:44:21.618 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-18T17:44:21.619 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-18T17:44:21.619 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10113 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-18T17:44:21.619 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 89473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-18T17:44:21.619 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9593 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-18T17:44:21.619 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 37129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-18T17:44:21.620 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 95417 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-18T17:44:21.620 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-18T17:44:21.620 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-18T17:44:21.620 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11857 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-18T17:44:21.620 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28929 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-18T17:44:21.620 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-18T17:44:21.621 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 44065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-18T17:44:21.621 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 615313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-18T17:44:21.621 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 42561 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-18T17:44:21.621 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-18T17:44:21.621 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30569 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-18T17:44:21.621 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 39785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-18T17:44:21.622 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-18T17:44:21.622 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49377 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-18T17:44:21.622 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10457 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-18T17:44:21.622 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-18T17:44:21.622 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 40721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-18T17:44:21.623 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-18T17:44:21.623 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-18T17:44:21.623 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-18T17:44:21.623 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14185 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-18T17:44:21.623 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-18T17:44:21.624 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-18T17:44:21.624 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-18T17:44:21.624 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14329 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-18T17:44:21.624 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13673 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-18T17:44:21.624 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11841 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_osf.ko 2023-10-18T17:44:21.624 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-18T17:44:21.625 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-18T17:44:21.625 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-18T17:44:21.625 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-18T17:44:21.625 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-18T17:44:21.625 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20369 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_synproxy.ko 2023-10-18T17:44:21.626 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tproxy.ko 2023-10-18T17:44:21.626 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 89313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/x_tables.ko 2023-10-18T17:44:21.626 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-18T17:44:21.626 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-18T17:44:21.626 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-18T17:44:21.626 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-18T17:44:21.627 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 18169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-18T17:44:21.627 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-18T17:44:21.627 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-18T17:44:21.627 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-18T17:44:21.627 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-18T17:44:21.628 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13889 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-18T17:44:21.628 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-18T17:44:21.628 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11753 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-18T17:44:21.628 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-18T17:44:21.628 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-18T17:44:21.628 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-18T17:44:21.629 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-18T17:44:21.629 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-18T17:44:21.629 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 14337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-18T17:44:21.629 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-18T17:44:21.629 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-18T17:44:21.630 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17361 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-18T17:44:21.630 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-18T17:44:21.631 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-18T17:44:21.631 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-18T17:44:21.631 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-18T17:44:21.632 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-18T17:44:21.632 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11209 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-18T17:44:21.632 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-18T17:44:21.633 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10465 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-18T17:44:21.633 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11081 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-18T17:44:21.633 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-18T17:44:21.633 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-18T17:44:21.633 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-18T17:44:21.633 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-18T17:44:21.634 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-18T17:44:21.634 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-18T17:44:21.634 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8801 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-18T17:44:21.634 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-18T17:44:21.634 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-18T17:44:21.635 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 43289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-18T17:44:21.635 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-18T17:44:21.635 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7865 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-18T17:44:21.635 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11921 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-18T17:44:21.635 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-18T17:44:21.635 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11785 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-18T17:44:21.636 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16041 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-18T17:44:21.636 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8001 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-18T17:44:21.636 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-18T17:44:21.636 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7545 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-18T17:44:21.636 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8641 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-18T17:44:21.637 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11825 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-18T17:44:21.637 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13817 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-18T17:44:21.637 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-18T17:44:21.637 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-18T17:44:21.638 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-18T17:44:21.638 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-18T17:44:21.638 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-18T17:44:21.638 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11985 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-18T17:44:21.638 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-18T17:44:21.639 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-18T17:44:21.639 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-18T17:44:21.639 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-18T17:44:21.639 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-18T17:44:21.639 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-18T17:44:21.639 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 16089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-18T17:44:21.640 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9505 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-18T17:44:21.640 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9473 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-18T17:44:21.640 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8809 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-18T17:44:21.640 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 8553 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-18T17:44:21.640 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15777 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-18T17:44:21.640 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11129 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-18T17:44:21.641 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-18T17:44:21.641 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/ 2023-10-18T17:44:21.641 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13969 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-18T17:44:21.641 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/ 2023-10-18T17:44:21.641 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 12425 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-18T17:44:21.642 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/ 2023-10-18T17:44:21.642 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 380033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-18T17:44:21.642 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10321 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-18T17:44:21.642 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-18T17:44:21.642 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 11793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-18T17:44:21.643 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/ 2023-10-18T17:44:21.643 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13057 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-18T17:44:21.643 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/ 2023-10-18T17:44:21.643 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 110201 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/phonet.ko 2023-10-18T17:44:21.644 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54697 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/phonet/pn_pep.ko 2023-10-18T17:44:21.644 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/ 2023-10-18T17:44:21.644 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 251481 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-18T17:44:21.644 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 164937 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-18T17:44:21.644 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 67377 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-18T17:44:21.644 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/ 2023-10-18T17:44:21.645 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 819657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rxrpc/rxrpc.ko 2023-10-18T17:44:21.645 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/ 2023-10-18T17:44:21.645 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32241 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-18T17:44:21.645 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23577 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-18T17:44:21.645 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26873 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-18T17:44:21.646 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34097 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-18T17:44:21.646 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-18T17:44:21.646 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 43449 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-18T17:44:21.646 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-18T17:44:21.646 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-18T17:44:21.647 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35297 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-18T17:44:21.647 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36665 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-18T17:44:21.647 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26169 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-18T17:44:21.647 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33961 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-18T17:44:21.647 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19233 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_cgroup.ko 2023-10-18T17:44:21.647 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31401 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-18T17:44:21.648 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31769 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-18T17:44:21.648 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 38489 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-18T17:44:21.648 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 61409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-18T17:44:21.648 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7441 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-18T17:44:21.648 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 13409 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-18T17:44:21.648 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33289 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-18T17:44:21.649 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-18T17:44:21.649 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-18T17:44:21.649 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7249 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-18T17:44:21.649 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 21649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-18T17:44:21.649 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20849 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-18T17:44:21.649 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 34633 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-18T17:44:21.650 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 33609 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-18T17:44:21.650 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31617 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq_codel.ko 2023-10-18T17:44:21.650 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 35433 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-18T17:44:21.650 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 49601 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-18T17:44:21.650 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22089 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-18T17:44:21.651 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 66393 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-18T17:44:21.651 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26073 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-18T17:44:21.651 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 30273 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-18T17:44:21.651 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-18T17:44:21.651 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26065 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-18T17:44:21.651 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 36137 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-18T17:44:21.652 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 27497 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-18T17:44:21.652 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10625 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-18T17:44:21.652 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 24729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-18T17:44:21.652 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 48793 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-18T17:44:21.652 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 32905 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-18T17:44:21.652 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 26881 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-18T17:44:21.653 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 31353 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-18T17:44:21.653 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28897 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-18T17:44:21.653 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 28105 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-18T17:44:21.653 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/ 2023-10-18T17:44:21.654 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 929745 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-18T17:44:21.654 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 19033 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-18T17:44:21.654 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/ 2023-10-18T17:44:21.654 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/ 2023-10-18T17:44:21.654 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 336529 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-18T17:44:21.655 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 54737 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-18T17:44:21.655 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 1520161 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-18T17:44:21.655 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/ 2023-10-18T17:44:21.655 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 707721 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-18T17:44:21.655 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/ 2023-10-18T17:44:21.655 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10025 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-18T17:44:21.656 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 862641 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-18T17:44:21.656 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/ 2023-10-18T17:44:21.656 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15729 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-18T17:44:21.656 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/ 2023-10-18T17:44:21.656 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 85337 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-18T17:44:21.656 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 68049 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-18T17:44:21.657 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 96193 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-18T17:44:21.657 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 10705 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-18T17:44:21.657 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 17521 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-18T17:44:21.657 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/ 2023-10-18T17:44:21.657 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 125177 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/x25/x25.ko 2023-10-18T17:44:21.658 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/ 2023-10-18T17:44:21.658 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 22657 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_algo.ko 2023-10-18T17:44:21.658 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 20649 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-18T17:44:21.658 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 102257 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_user.ko 2023-10-18T17:44:21.658 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/ 2023-10-18T17:44:21.658 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/ 2023-10-18T17:44:21.659 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 15537 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-18T17:44:21.659 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 639863 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-18T17:44:21.659 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 623919 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-18T17:44:21.659 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 7663 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-18T17:44:21.659 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 23428 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-18T17:44:21.660 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 9821 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-18T17:44:21.660 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 70004 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-18T17:44:21.660 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 146313 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-18T17:44:21.660 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 232548 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-18T17:44:21.660 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 210 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-18T17:44:21.660 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 72606 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-18T17:44:21.661 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 879 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-18T17:44:21.661 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 254438 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-18T17:44:21.661 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 309363 2023-10-10 19:59 ./lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-18T17:44:21.661 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./usr/ 2023-10-18T17:44:21.661 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./usr/share/ 2023-10-18T17:44:21.661 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./usr/share/doc/ 2023-10-18T17:44:21.662 INFO:teuthology.orchestra.run.smithi072.stdout:drwxr-xr-x root/root 0 2023-10-10 19:59 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/ 2023-10-18T17:44:21.662 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 174 2023-10-10 19:59 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/changelog.Debian.gz 2023-10-18T17:44:21.662 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r-- root/root 691 2023-10-10 19:59 ./usr/share/doc/linux-image-6.6.0-rc5-g47672119149b/copyright 2023-10-18T17:44:21.663 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-18T17:44:21.663 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g47672119149b 2023-10-18T17:44:21.664 DEBUG:teuthology.orchestra.run.smithi072:> egrep '(submenu|menuentry.*6.6.0-rc5-g47672119149b).*{' /boot/grub/grub.cfg 2023-10-18T17:44:21.672 INFO:teuthology.orchestra.run.smithi072.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-18T17:44:21.673 INFO:teuthology.orchestra.run.smithi072.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-18T17:44:21.673 INFO:teuthology.orchestra.run.smithi072.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g47672119149b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g47672119149b-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-18T17:44:21.674 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-18T17:44:21.674 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g47672119149b 2023-10-18T17:44:21.674 DEBUG:teuthology.orchestra.run.smithi072:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-18T17:44:21.677 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi072.front.sepia.ceph.com to complete... 2023-10-18T17:44:22.474 INFO:teuthology.orchestra.run.smithi072.stderr:Sourcing file `/etc/default/grub' 2023-10-18T17:44:22.483 INFO:teuthology.orchestra.run.smithi072.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-18T17:44:22.541 INFO:teuthology.orchestra.run.smithi072.stderr:Generating grub configuration file ... 2023-10-18T17:44:22.818 INFO:teuthology.orchestra.run.smithi072.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T17:44:22.830 INFO:teuthology.orchestra.run.smithi072.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g47672119149b 2023-10-18T17:44:23.178 INFO:teuthology.orchestra.run.smithi072.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-18T17:44:23.190 INFO:teuthology.orchestra.run.smithi072.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-18T17:44:24.833 INFO:teuthology.orchestra.run.smithi072.stderr:done 2023-10-18T17:44:55.969 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-18T17:44:55.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-10-18T17:44:55.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T17:45:30.490 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-10-18T17:45:39.530 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-10-18T17:45:39.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T17:45:57.889 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-10-18T17:46:09.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-10-18T17:46:09.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T17:46:12.429 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-10-18T17:46:15.670 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-10-18T17:46:15.671 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g47672119149b"... 2023-10-18T17:46:15.671 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-10-18T17:46:15.729 INFO:teuthology.orchestra.run.smithi072.stdout:6.6.0-rc5-g47672119149b 2023-10-18T17:46:15.730 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-18T17:46:15.730 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T17:46:15.731 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-18T17:46:16.731 DEBUG:teuthology.parallel:result is None 2023-10-18T17:46:16.732 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-18T17:46:16.747 INFO:teuthology.task.internal:Creating test directory... 2023-10-18T17:46:16.747 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T17:46:16.755 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T17:46:16.766 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-18T17:46:16.773 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-18T17:46:16.781 INFO:teuthology.task.internal:Creating archive directory... 2023-10-18T17:46:16.782 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T17:46:16.821 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T17:46:16.915 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-18T17:46:16.923 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-18T17:46:16.923 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T17:46:16.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:46:16.929 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T17:46:16.934 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T17:46:17.000 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T17:46:17.016 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T17:46:17.220 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T17:46:17.268 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T17:46:17.273 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-18T17:46:17.282 INFO:teuthology.task.internal:Configuring sudo... 2023-10-18T17:46:17.282 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T17:46:17.289 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T17:46:17.319 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-18T17:46:17.329 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-18T17:46:17.329 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T17:46:17.370 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T17:46:17.378 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T17:46:17.439 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T17:46:17.501 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:46:17.502 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T17:46:17.597 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T17:46:17.605 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T17:46:17.612 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:46:17.612 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T17:46:17.638 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2023-10-18T17:46:17.645 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2023-10-18T17:46:17.764 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-18T17:46:17.774 INFO:teuthology.task.internal:Starting timer... 2023-10-18T17:46:17.775 INFO:teuthology.run_tasks:Running task pcp... 2023-10-18T17:46:17.792 INFO:teuthology.run_tasks:Running task selinux... 2023-10-18T17:46:17.807 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-18T17:46:17.808 DEBUG:teuthology.task.selinux:Excluding smithi072: OS 'ubuntu' does not support SELinux 2023-10-18T17:46:17.808 DEBUG:teuthology.task.selinux:Excluding smithi179: OS 'ubuntu' does not support SELinux 2023-10-18T17:46:17.808 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-18T17:46:17.809 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-18T17:46:17.809 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-18T17:46:17.809 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-18T17:46:17.821 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-18T17:46:17.826 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-18T17:46:17.940 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-18T17:46:17.955 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-18T17:46:17.956 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi072.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-18T17:54:15.631 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2023-10-18T17:54:15.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-10-18T17:54:15.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T17:54:15.742 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-10-18T17:54:15.972 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-10-18T17:54:15.973 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2023-10-18T17:54:15.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T17:54:16.021 DEBUG:teuthology.orchestra.run.smithi179:> true 2023-10-18T17:54:16.145 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2023-10-18T17:54:16.145 INFO:teuthology.run_tasks:Running task clock... 2023-10-18T17:54:16.156 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-18T17:54:16.157 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T17:54:16.158 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T17:54:16.165 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T17:54:16.165 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T17:54:16.223 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-18T17:54:16.223 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: Command line: ntpd -gq 2023-10-18T17:54:16.224 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: proto: precision = 0.280 usec (-22) 2023-10-18T17:54:16.224 INFO:teuthology.orchestra.run.smithi179.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-18T17:54:16.225 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-18T17:54:16.225 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: restrict ::: KOD does nothing without LIMITED. 2023-10-18T17:54:16.225 INFO:teuthology.orchestra.run.smithi179.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-18T17:54:16.226 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: Listen and drop on 0 v6wildcard [::]:123 2023-10-18T17:54:16.227 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-18T17:54:16.227 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: Listen normally on 2 lo 127.0.0.1:123 2023-10-18T17:54:16.227 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: Listen normally on 3 enp3s0f1 172.21.15.179:123 2023-10-18T17:54:16.227 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: Listen normally on 4 lo [::1]:123 2023-10-18T17:54:16.228 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:725b%5]:123 2023-10-18T17:54:16.228 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:16 ntpd[23239]: Listening on routing socket on fd #22 for interface updates 2023-10-18T17:54:16.234 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-18T17:54:16.235 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: Command line: ntpd -gq 2023-10-18T17:54:16.236 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: proto: precision = 0.051 usec (-24) 2023-10-18T17:54:16.236 INFO:teuthology.orchestra.run.smithi072.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-18T17:54:16.237 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-18T17:54:16.237 INFO:teuthology.orchestra.run.smithi072.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-18T17:54:16.238 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: restrict ::: KOD does nothing without LIMITED. 2023-10-18T17:54:16.239 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: Listen and drop on 0 v6wildcard [::]:123 2023-10-18T17:54:16.239 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-18T17:54:16.239 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: Listen normally on 2 lo 127.0.0.1:123 2023-10-18T17:54:16.239 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: Listen normally on 3 enp3s0f1 172.21.15.72:123 2023-10-18T17:54:16.239 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: Listen normally on 4 lo [::1]:123 2023-10-18T17:54:16.240 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:b765%5]:123 2023-10-18T17:54:16.240 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:16 ntpd[22831]: Listening on routing socket on fd #22 for interface updates 2023-10-18T17:54:17.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:17.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:17.227 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.227 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.227 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:17.227 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:17 ntpd[23239]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:17.236 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.236 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.236 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.236 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:17.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:17.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:17.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:17.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:17.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:17.241 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:17.241 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:17 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:19.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:19.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:19.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:19.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:19 ntpd[23239]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:19.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:19.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:19.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:19.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:19.241 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:19.241 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:19.242 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:19.242 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:19 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:20.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:20 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:20.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:20 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:20.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:20 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:20.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:20 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:20.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:20 ntpd[23239]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:20.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:20 ntpd[23239]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:21.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:21.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:21.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.227 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.227 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:21.227 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:21 ntpd[23239]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:21.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.239 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:21.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:21.242 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:21.243 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:21.243 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:21.243 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:21 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:22.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:22 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:22.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:22 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:22.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:22 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:22 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:22 ntpd[23239]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:22.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:22 ntpd[23239]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:23.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:23.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:23.224 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:23.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:23.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:23.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:23.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:23.225 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:23.226 INFO:teuthology.orchestra.run.smithi179.stdout:18 Oct 17:54:23 ntpd[23239]: ntpd: time slew +0.000896 s 2023-10-18T17:54:23.226 INFO:teuthology.orchestra.run.smithi179.stdout:ntpd: time slew +0.000896s 2023-10-18T17:54:23.226 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-18T17:54:23.227 INFO:teuthology.orchestra.run.smithi179.stderr:18 Oct 17:54:23 ntpd[23239]: can't open /var/log/ntpstats/loopstats.20231018: Permission denied 2023-10-18T17:54:23.236 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:23.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:23.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:23.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:23.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:23.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:23.237 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-18T17:54:23.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: can't open /var/log/ntpstats/rawstats.20231018: Permission denied 2023-10-18T17:54:23.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-18T17:54:23.238 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: can't open /var/log/ntpstats/peerstats.20231018: Permission denied 2023-10-18T17:54:23.239 INFO:teuthology.orchestra.run.smithi072.stdout:18 Oct 17:54:23 ntpd[22831]: ntpd: time slew +0.002207 s 2023-10-18T17:54:23.239 INFO:teuthology.orchestra.run.smithi072.stdout:ntpd: time slew +0.002207s 2023-10-18T17:54:23.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-18T17:54:23.240 INFO:teuthology.orchestra.run.smithi072.stderr:18 Oct 17:54:23 ntpd[22831]: can't open /var/log/ntpstats/loopstats.20231018: Permission denied 2023-10-18T17:54:23.293 INFO:teuthology.orchestra.run.smithi179.stdout: remote refid st t when poll reach delay offset jitter 2023-10-18T17:54:23.294 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================== 2023-10-18T17:54:23.294 INFO:teuthology.orchestra.run.smithi179.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-18T17:54:23.294 INFO:teuthology.orchestra.run.smithi179.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-18T17:54:23.294 INFO:teuthology.orchestra.run.smithi179.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-18T17:54:23.294 INFO:teuthology.orchestra.run.smithi179.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-18T17:54:23.367 INFO:teuthology.orchestra.run.smithi072.stdout: remote refid st t when poll reach delay offset jitter 2023-10-18T17:54:23.368 INFO:teuthology.orchestra.run.smithi072.stdout:============================================================================== 2023-10-18T17:54:23.368 INFO:teuthology.orchestra.run.smithi072.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-18T17:54:23.368 INFO:teuthology.orchestra.run.smithi072.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-18T17:54:23.368 INFO:teuthology.orchestra.run.smithi072.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-18T17:54:23.368 INFO:teuthology.orchestra.run.smithi072.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-18T17:54:23.369 INFO:teuthology.run_tasks:Running task install... 2023-10-18T17:54:23.378 DEBUG:teuthology.task.install:project ceph 2023-10-18T17:54:23.379 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5'}} 2023-10-18T17:54:23.379 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5'} 2023-10-18T17:54:23.380 INFO:teuthology.task.install:Using flavor: default 2023-10-18T17:54:23.385 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-18T17:54:23.386 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-18T17:54:23.387 DEBUG:teuthology.orchestra.run.smithi072:> sudo apt-key list | grep Ceph 2023-10-18T17:54:23.387 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-key list | grep Ceph 2023-10-18T17:54:23.496 INFO:teuthology.orchestra.run.smithi179.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-18T17:54:23.553 INFO:teuthology.orchestra.run.smithi179.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-18T17:54:23.554 INFO:teuthology.orchestra.run.smithi179.stdout:uid [ unknown] Ceph.com (release key) 2023-10-18T17:54:23.555 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-18T17:54:23.555 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-18T17:54:23.555 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 2023-10-18T17:54:23.623 INFO:teuthology.orchestra.run.smithi072.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-18T17:54:23.730 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default/ 2023-10-18T17:54:23.730 INFO:teuthology.task.install.deb:Package version is 16.2.14-100-gb4ddd52c-1focal 2023-10-18T17:54:23.762 INFO:teuthology.orchestra.run.smithi072.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-18T17:54:23.762 INFO:teuthology.orchestra.run.smithi072.stdout:uid [ unknown] Ceph.com (release key) 2023-10-18T17:54:23.763 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-18T17:54:23.764 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-18T17:54:23.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 2023-10-18T17:54:23.845 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:54:23.846 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-18T17:54:23.863 DEBUG:teuthology.orchestra.run.smithi179:> sudo apt-get update 2023-10-18T17:54:23.998 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default/ 2023-10-18T17:54:23.999 INFO:teuthology.task.install.deb:Package version is 16.2.14-100-gb4ddd52c-1focal 2023-10-18T17:54:24.047 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-18T17:54:24.054 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-18T17:54:24.071 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-18T17:54:24.164 INFO:teuthology.orchestra.run.smithi179.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-18T17:54:24.195 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:54:24.195 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-18T17:54:24.196 INFO:teuthology.orchestra.run.smithi179.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal InRelease 2023-10-18T17:54:24.231 DEBUG:teuthology.orchestra.run.smithi072:> sudo apt-get update 2023-10-18T17:54:24.232 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal Release [7,680 B] 2023-10-18T17:54:24.270 INFO:teuthology.orchestra.run.smithi179.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal Release.gpg 2023-10-18T17:54:24.308 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main i386 Packages [3,084 B] 2023-10-18T17:54:24.356 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 Packages [17.2 kB] 2023-10-18T17:54:24.514 INFO:teuthology.orchestra.run.smithi072.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-18T17:54:24.515 INFO:teuthology.orchestra.run.smithi072.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-18T17:54:24.515 INFO:teuthology.orchestra.run.smithi072.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-18T17:54:24.530 INFO:teuthology.orchestra.run.smithi072.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-18T17:54:24.561 INFO:teuthology.orchestra.run.smithi072.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal InRelease 2023-10-18T17:54:24.583 INFO:teuthology.orchestra.run.smithi072.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal Release [7,680 B] 2023-10-18T17:54:24.609 INFO:teuthology.orchestra.run.smithi072.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal Release.gpg 2023-10-18T17:54:24.631 INFO:teuthology.orchestra.run.smithi072.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main i386 Packages [3,084 B] 2023-10-18T17:54:24.675 INFO:teuthology.orchestra.run.smithi072.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 Packages [17.2 kB] 2023-10-18T17:54:24.693 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 27.9 kB in 1s (45.0 kB/s) 2023-10-18T17:54:25.729 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2023-10-18T17:54:25.754 DEBUG:teuthology.orchestra.run.smithi179:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-100-gb4ddd52c-1focal cephadm=16.2.14-100-gb4ddd52c-1focal ceph-mds=16.2.14-100-gb4ddd52c-1focal ceph-mgr=16.2.14-100-gb4ddd52c-1focal ceph-common=16.2.14-100-gb4ddd52c-1focal ceph-fuse=16.2.14-100-gb4ddd52c-1focal ceph-test=16.2.14-100-gb4ddd52c-1focal radosgw=16.2.14-100-gb4ddd52c-1focal python3-rados=16.2.14-100-gb4ddd52c-1focal python3-rgw=16.2.14-100-gb4ddd52c-1focal python3-cephfs=16.2.14-100-gb4ddd52c-1focal python3-rbd=16.2.14-100-gb4ddd52c-1focal libcephfs2=16.2.14-100-gb4ddd52c-1focal libcephfs-dev=16.2.14-100-gb4ddd52c-1focal librados2=16.2.14-100-gb4ddd52c-1focal librbd1=16.2.14-100-gb4ddd52c-1focal rbd-fuse=16.2.14-100-gb4ddd52c-1focal python3-cephfs=16.2.14-100-gb4ddd52c-1focal cephfs-shell=16.2.14-100-gb4ddd52c-1focal cephfs-top=16.2.14-100-gb4ddd52c-1focal cephfs-mirror=16.2.14-100-gb4ddd52c-1focal 2023-10-18T17:54:25.825 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2023-10-18T17:54:25.950 INFO:teuthology.orchestra.run.smithi072.stdout:Fetched 27.9 kB in 1s (20.2 kB/s) 2023-10-18T17:54:26.005 INFO:teuthology.orchestra.run.smithi179.stdout:Building dependency tree... 2023-10-18T17:54:26.005 INFO:teuthology.orchestra.run.smithi179.stdout:Reading state information... 2023-10-18T17:54:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages were automatically installed and are no longer required: 2023-10-18T17:54:26.172 INFO:teuthology.orchestra.run.smithi179.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-18T17:54:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-18T17:54:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:The following additional packages will be installed: 2023-10-18T17:54:26.173 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-18T17:54:26.174 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2023-10-18T17:54:26.174 INFO:teuthology.orchestra.run.smithi179.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2023-10-18T17:54:26.174 INFO:teuthology.orchestra.run.smithi179.stdout: liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 2023-10-18T17:54:26.174 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2023-10-18T17:54:26.175 INFO:teuthology.orchestra.run.smithi179.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2023-10-18T17:54:26.175 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2023-10-18T17:54:26.175 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2023-10-18T17:54:26.175 INFO:teuthology.orchestra.run.smithi179.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-10-18T17:54:26.175 INFO:teuthology.orchestra.run.smithi179.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-10-18T17:54:26.175 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2023-10-18T17:54:26.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-18T17:54:26.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-18T17:54:26.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-18T17:54:26.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-18T17:54:26.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-18T17:54:26.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-18T17:54:26.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-18T17:54:26.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2023-10-18T17:54:26.177 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet 2023-10-18T17:54:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:Suggested packages: 2023-10-18T17:54:26.178 INFO:teuthology.orchestra.run.smithi179.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-18T17:54:26.178 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-18T17:54:26.178 INFO:teuthology.orchestra.run.smithi179.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-18T17:54:26.178 INFO:teuthology.orchestra.run.smithi179.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-18T17:54:26.178 INFO:teuthology.orchestra.run.smithi179.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-18T17:54:26.179 INFO:teuthology.orchestra.run.smithi179.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-18T17:54:26.179 INFO:teuthology.orchestra.run.smithi179.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-18T17:54:26.179 INFO:teuthology.orchestra.run.smithi179.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-18T17:54:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:Recommended packages: 2023-10-18T17:54:26.179 INFO:teuthology.orchestra.run.smithi179.stdout: btrfs-tools 2023-10-18T17:54:26.290 INFO:teuthology.orchestra.run.smithi179.stdout:The following NEW packages will be installed: 2023-10-18T17:54:26.291 INFO:teuthology.orchestra.run.smithi179.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-18T17:54:26.291 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-18T17:54:26.291 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm cephfs-mirror 2023-10-18T17:54:26.292 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 libjaeger libjq1 2023-10-18T17:54:26.292 INFO:teuthology.orchestra.run.smithi179.stdout: libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d liblttng-ust0 2023-10-18T17:54:26.293 INFO:teuthology.orchestra.run.smithi179.stdout: liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2023-10-18T17:54:26.293 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2023-10-18T17:54:26.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2023-10-18T17:54:26.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-18T17:54:26.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2023-10-18T17:54:26.293 INFO:teuthology.orchestra.run.smithi179.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-10-18T17:54:26.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-10-18T17:54:26.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2023-10-18T17:54:26.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-18T17:54:26.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-18T17:54:26.294 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-18T17:54:26.295 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-18T17:54:26.295 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-18T17:54:26.295 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-18T17:54:26.295 INFO:teuthology.orchestra.run.smithi179.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-18T17:54:26.295 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-18T17:54:26.295 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug radosgw 2023-10-18T17:54:26.296 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse socat xmlstarlet 2023-10-18T17:54:26.296 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages will be upgraded: 2023-10-18T17:54:26.296 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 librbd1 2023-10-18T17:54:26.389 INFO:teuthology.orchestra.run.smithi179.stdout:2 upgraded, 104 newly installed, 0 to remove and 256 not upgraded. 2023-10-18T17:54:26.390 INFO:teuthology.orchestra.run.smithi179.stdout:Need to get 174 MB of archives. 2023-10-18T17:54:26.390 INFO:teuthology.orchestra.run.smithi179.stdout:After this operation, 805 MB of additional disk space will be used. 2023-10-18T17:54:26.390 INFO:teuthology.orchestra.run.smithi179.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-18T17:54:26.426 INFO:teuthology.orchestra.run.smithi179.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-18T17:54:26.441 INFO:teuthology.orchestra.run.smithi179.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-18T17:54:26.489 INFO:teuthology.orchestra.run.smithi179.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-18T17:54:26.547 INFO:teuthology.orchestra.run.smithi179.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libjaeger amd64 16.2.14-100-gb4ddd52c-1focal [3,964 B] 2023-10-18T17:54:26.563 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-18T17:54:26.567 INFO:teuthology.orchestra.run.smithi179.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-18T17:54:26.592 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-18T17:54:26.598 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-18T17:54:26.650 INFO:teuthology.orchestra.run.smithi179.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2023-10-18T17:54:26.662 INFO:teuthology.orchestra.run.smithi179.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-18T17:54:26.664 INFO:teuthology.orchestra.run.smithi179.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-18T17:54:26.677 INFO:teuthology.orchestra.run.smithi179.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-18T17:54:26.679 INFO:teuthology.orchestra.run.smithi179.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-18T17:54:26.680 INFO:teuthology.orchestra.run.smithi179.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.2.14-100-gb4ddd52c-1focal [3,382 kB] 2023-10-18T17:54:26.681 INFO:teuthology.orchestra.run.smithi179.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-18T17:54:26.699 INFO:teuthology.orchestra.run.smithi179.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-18T17:54:26.700 INFO:teuthology.orchestra.run.smithi179.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-18T17:54:26.700 INFO:teuthology.orchestra.run.smithi179.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-18T17:54:26.716 INFO:teuthology.orchestra.run.smithi179.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-18T17:54:26.739 INFO:teuthology.orchestra.run.smithi179.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-18T17:54:26.739 INFO:teuthology.orchestra.run.smithi179.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-18T17:54:26.741 INFO:teuthology.orchestra.run.smithi179.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-18T17:54:26.742 INFO:teuthology.orchestra.run.smithi179.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-18T17:54:26.743 INFO:teuthology.orchestra.run.smithi179.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-18T17:54:26.745 INFO:teuthology.orchestra.run.smithi179.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-18T17:54:26.746 INFO:teuthology.orchestra.run.smithi179.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-18T17:54:26.747 INFO:teuthology.orchestra.run.smithi179.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-18T17:54:26.754 INFO:teuthology.orchestra.run.smithi179.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-18T17:54:26.771 INFO:teuthology.orchestra.run.smithi179.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-18T17:54:26.772 INFO:teuthology.orchestra.run.smithi179.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-18T17:54:26.773 INFO:teuthology.orchestra.run.smithi179.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-18T17:54:26.775 INFO:teuthology.orchestra.run.smithi179.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-18T17:54:26.780 INFO:teuthology.orchestra.run.smithi179.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-18T17:54:26.782 INFO:teuthology.orchestra.run.smithi179.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-18T17:54:26.782 INFO:teuthology.orchestra.run.smithi179.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-18T17:54:26.785 INFO:teuthology.orchestra.run.smithi179.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-18T17:54:26.790 INFO:teuthology.orchestra.run.smithi179.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-18T17:54:26.793 INFO:teuthology.orchestra.run.smithi179.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-18T17:54:26.807 INFO:teuthology.orchestra.run.smithi179.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-18T17:54:26.809 INFO:teuthology.orchestra.run.smithi179.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-18T17:54:27.018 INFO:teuthology.orchestra.run.smithi179.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-18T17:54:27.060 INFO:teuthology.orchestra.run.smithi179.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-18T17:54:27.063 INFO:teuthology.orchestra.run.smithi179.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-18T17:54:27.141 INFO:teuthology.orchestra.run.smithi072.stdout:Reading package lists... 2023-10-18T17:54:27.158 INFO:teuthology.orchestra.run.smithi179.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-18T17:54:27.159 INFO:teuthology.orchestra.run.smithi179.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-18T17:54:27.160 INFO:teuthology.orchestra.run.smithi179.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-18T17:54:27.161 INFO:teuthology.orchestra.run.smithi179.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-18T17:54:27.161 INFO:teuthology.orchestra.run.smithi179.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-18T17:54:27.162 INFO:teuthology.orchestra.run.smithi179.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-18T17:54:27.163 INFO:teuthology.orchestra.run.smithi179.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-18T17:54:27.164 INFO:teuthology.orchestra.run.smithi179.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-18T17:54:27.165 INFO:teuthology.orchestra.run.smithi179.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-18T17:54:27.178 INFO:teuthology.orchestra.run.smithi179.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-18T17:54:27.182 DEBUG:teuthology.orchestra.run.smithi072:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-100-gb4ddd52c-1focal cephadm=16.2.14-100-gb4ddd52c-1focal ceph-mds=16.2.14-100-gb4ddd52c-1focal ceph-mgr=16.2.14-100-gb4ddd52c-1focal ceph-common=16.2.14-100-gb4ddd52c-1focal ceph-fuse=16.2.14-100-gb4ddd52c-1focal ceph-test=16.2.14-100-gb4ddd52c-1focal radosgw=16.2.14-100-gb4ddd52c-1focal python3-rados=16.2.14-100-gb4ddd52c-1focal python3-rgw=16.2.14-100-gb4ddd52c-1focal python3-cephfs=16.2.14-100-gb4ddd52c-1focal python3-rbd=16.2.14-100-gb4ddd52c-1focal libcephfs2=16.2.14-100-gb4ddd52c-1focal libcephfs-dev=16.2.14-100-gb4ddd52c-1focal librados2=16.2.14-100-gb4ddd52c-1focal librbd1=16.2.14-100-gb4ddd52c-1focal rbd-fuse=16.2.14-100-gb4ddd52c-1focal python3-cephfs=16.2.14-100-gb4ddd52c-1focal cephfs-shell=16.2.14-100-gb4ddd52c-1focal cephfs-top=16.2.14-100-gb4ddd52c-1focal cephfs-mirror=16.2.14-100-gb4ddd52c-1focal 2023-10-18T17:54:27.183 INFO:teuthology.orchestra.run.smithi179.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-18T17:54:27.198 INFO:teuthology.orchestra.run.smithi179.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-18T17:54:27.200 INFO:teuthology.orchestra.run.smithi179.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-18T17:54:27.201 INFO:teuthology.orchestra.run.smithi179.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-18T17:54:27.205 INFO:teuthology.orchestra.run.smithi179.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-18T17:54:27.208 INFO:teuthology.orchestra.run.smithi179.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-18T17:54:27.209 INFO:teuthology.orchestra.run.smithi179.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-18T17:54:27.213 INFO:teuthology.orchestra.run.smithi179.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-18T17:54:27.213 INFO:teuthology.orchestra.run.smithi179.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-18T17:54:27.219 INFO:teuthology.orchestra.run.smithi179.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-18T17:54:27.219 INFO:teuthology.orchestra.run.smithi179.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-18T17:54:27.225 INFO:teuthology.orchestra.run.smithi179.stdout:Get:66 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.2.14-100-gb4ddd52c-1focal [3,389 kB] 2023-10-18T17:54:27.238 INFO:teuthology.orchestra.run.smithi179.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-18T17:54:27.258 INFO:teuthology.orchestra.run.smithi179.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-18T17:54:27.258 INFO:teuthology.orchestra.run.smithi179.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-18T17:54:27.259 INFO:teuthology.orchestra.run.smithi179.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-18T17:54:27.259 INFO:teuthology.orchestra.run.smithi179.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-18T17:54:27.260 INFO:teuthology.orchestra.run.smithi179.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-18T17:54:27.260 INFO:teuthology.orchestra.run.smithi179.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-18T17:54:27.261 INFO:teuthology.orchestra.run.smithi179.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-18T17:54:27.262 INFO:teuthology.orchestra.run.smithi179.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-18T17:54:27.310 INFO:teuthology.orchestra.run.smithi072.stdout:Reading package lists... 2023-10-18T17:54:27.511 INFO:teuthology.orchestra.run.smithi072.stdout:Building dependency tree... 2023-10-18T17:54:27.511 INFO:teuthology.orchestra.run.smithi072.stdout:Reading state information... 2023-10-18T17:54:27.563 INFO:teuthology.orchestra.run.smithi179.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.2.14-100-gb4ddd52c-1focal [729 kB] 2023-10-18T17:54:27.633 INFO:teuthology.orchestra.run.smithi179.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.2.14-100-gb4ddd52c-1focal [413 kB] 2023-10-18T17:54:27.669 INFO:teuthology.orchestra.run.smithi179.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.2.14-100-gb4ddd52c-1focal [22.2 kB] 2023-10-18T17:54:27.670 INFO:teuthology.orchestra.run.smithi179.stdout:Get:79 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.2.14-100-gb4ddd52c-1focal [212 kB] 2023-10-18T17:54:27.690 INFO:teuthology.orchestra.run.smithi072.stdout:The following packages were automatically installed and are no longer required: 2023-10-18T17:54:27.690 INFO:teuthology.orchestra.run.smithi072.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-18T17:54:27.691 INFO:teuthology.orchestra.run.smithi072.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-18T17:54:27.691 INFO:teuthology.orchestra.run.smithi072.stdout:The following additional packages will be installed: 2023-10-18T17:54:27.691 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-18T17:54:27.692 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2023-10-18T17:54:27.692 INFO:teuthology.orchestra.run.smithi072.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2023-10-18T17:54:27.692 INFO:teuthology.orchestra.run.smithi072.stdout: liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 2023-10-18T17:54:27.692 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2023-10-18T17:54:27.693 INFO:teuthology.orchestra.run.smithi072.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2023-10-18T17:54:27.693 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2023-10-18T17:54:27.693 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2023-10-18T17:54:27.693 INFO:teuthology.orchestra.run.smithi072.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-10-18T17:54:27.693 INFO:teuthology.orchestra.run.smithi072.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-10-18T17:54:27.694 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2023-10-18T17:54:27.694 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-18T17:54:27.694 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-18T17:54:27.694 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-18T17:54:27.694 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-18T17:54:27.695 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-18T17:54:27.695 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-18T17:54:27.695 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-18T17:54:27.695 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2023-10-18T17:54:27.695 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet 2023-10-18T17:54:27.696 INFO:teuthology.orchestra.run.smithi072.stdout:Suggested packages: 2023-10-18T17:54:27.696 INFO:teuthology.orchestra.run.smithi072.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-18T17:54:27.696 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-18T17:54:27.696 INFO:teuthology.orchestra.run.smithi072.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-18T17:54:27.697 INFO:teuthology.orchestra.run.smithi072.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-18T17:54:27.697 INFO:teuthology.orchestra.run.smithi072.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-18T17:54:27.697 INFO:teuthology.orchestra.run.smithi072.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-18T17:54:27.697 INFO:teuthology.orchestra.run.smithi072.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-18T17:54:27.697 INFO:teuthology.orchestra.run.smithi072.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-18T17:54:27.698 INFO:teuthology.orchestra.run.smithi072.stdout:Recommended packages: 2023-10-18T17:54:27.698 INFO:teuthology.orchestra.run.smithi072.stdout: btrfs-tools 2023-10-18T17:54:27.698 INFO:teuthology.orchestra.run.smithi179.stdout:Get:80 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.2.14-100-gb4ddd52c-1focal [38.4 kB] 2023-10-18T17:54:27.699 INFO:teuthology.orchestra.run.smithi179.stdout:Get:81 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.2.14-100-gb4ddd52c-1focal [387 kB] 2023-10-18T17:54:27.734 INFO:teuthology.orchestra.run.smithi179.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.2.14-100-gb4ddd52c-1focal [3,689 kB] 2023-10-18T17:54:27.885 INFO:teuthology.orchestra.run.smithi072.stdout:The following NEW packages will be installed: 2023-10-18T17:54:27.885 INFO:teuthology.orchestra.run.smithi072.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-18T17:54:27.886 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-18T17:54:27.886 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm cephfs-mirror 2023-10-18T17:54:27.887 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 libjaeger libjq1 2023-10-18T17:54:27.887 INFO:teuthology.orchestra.run.smithi072.stdout: libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d liblttng-ust0 2023-10-18T17:54:27.887 INFO:teuthology.orchestra.run.smithi072.stdout: liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2023-10-18T17:54:27.887 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2023-10-18T17:54:27.888 INFO:teuthology.orchestra.run.smithi072.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2023-10-18T17:54:27.888 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-18T17:54:27.888 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2023-10-18T17:54:27.888 INFO:teuthology.orchestra.run.smithi072.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-10-18T17:54:27.888 INFO:teuthology.orchestra.run.smithi072.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-10-18T17:54:27.889 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2023-10-18T17:54:27.889 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-18T17:54:27.889 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-18T17:54:27.889 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-18T17:54:27.889 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-18T17:54:27.890 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-18T17:54:27.890 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-18T17:54:27.890 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-18T17:54:27.890 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-18T17:54:27.890 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug radosgw 2023-10-18T17:54:27.891 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse socat xmlstarlet 2023-10-18T17:54:27.891 INFO:teuthology.orchestra.run.smithi072.stdout:The following packages will be upgraded: 2023-10-18T17:54:27.891 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 librbd1 2023-10-18T17:54:27.985 INFO:teuthology.orchestra.run.smithi072.stdout:2 upgraded, 104 newly installed, 0 to remove and 256 not upgraded. 2023-10-18T17:54:27.985 INFO:teuthology.orchestra.run.smithi072.stdout:Need to get 174 MB of archives. 2023-10-18T17:54:27.985 INFO:teuthology.orchestra.run.smithi072.stdout:After this operation, 805 MB of additional disk space will be used. 2023-10-18T17:54:27.986 INFO:teuthology.orchestra.run.smithi072.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-18T17:54:28.004 INFO:teuthology.orchestra.run.smithi072.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-18T17:54:28.013 INFO:teuthology.orchestra.run.smithi072.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-18T17:54:28.040 INFO:teuthology.orchestra.run.smithi072.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-18T17:54:28.068 INFO:teuthology.orchestra.run.smithi072.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-18T17:54:28.069 INFO:teuthology.orchestra.run.smithi072.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-18T17:54:28.079 INFO:teuthology.orchestra.run.smithi072.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-18T17:54:28.081 INFO:teuthology.orchestra.run.smithi072.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-18T17:54:28.088 INFO:teuthology.orchestra.run.smithi179.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.2.14-100-gb4ddd52c-1focal [116 kB] 2023-10-18T17:54:28.099 INFO:teuthology.orchestra.run.smithi072.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2023-10-18T17:54:28.103 INFO:teuthology.orchestra.run.smithi072.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-18T17:54:28.103 INFO:teuthology.orchestra.run.smithi072.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-18T17:54:28.109 INFO:teuthology.orchestra.run.smithi072.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-18T17:54:28.109 INFO:teuthology.orchestra.run.smithi072.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-18T17:54:28.111 INFO:teuthology.orchestra.run.smithi072.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-18T17:54:28.113 INFO:teuthology.orchestra.run.smithi072.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-18T17:54:28.113 INFO:teuthology.orchestra.run.smithi072.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-18T17:54:28.114 INFO:teuthology.orchestra.run.smithi072.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-18T17:54:28.121 INFO:teuthology.orchestra.run.smithi072.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-18T17:54:28.125 INFO:teuthology.orchestra.run.smithi179.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.2.14-100-gb4ddd52c-1focal [448 kB] 2023-10-18T17:54:28.132 INFO:teuthology.orchestra.run.smithi072.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-18T17:54:28.133 INFO:teuthology.orchestra.run.smithi072.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-18T17:54:28.133 INFO:teuthology.orchestra.run.smithi072.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-18T17:54:28.136 INFO:teuthology.orchestra.run.smithi072.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-18T17:54:28.138 INFO:teuthology.orchestra.run.smithi072.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-18T17:54:28.140 INFO:teuthology.orchestra.run.smithi072.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-18T17:54:28.142 INFO:teuthology.orchestra.run.smithi072.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-18T17:54:28.143 INFO:teuthology.orchestra.run.smithi072.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-18T17:54:28.147 INFO:teuthology.orchestra.run.smithi072.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-18T17:54:28.154 INFO:teuthology.orchestra.run.smithi072.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-18T17:54:28.155 INFO:teuthology.orchestra.run.smithi072.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-18T17:54:28.156 INFO:teuthology.orchestra.run.smithi072.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-18T17:54:28.171 INFO:teuthology.orchestra.run.smithi072.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-18T17:54:28.176 INFO:teuthology.orchestra.run.smithi072.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-18T17:54:28.177 INFO:teuthology.orchestra.run.smithi072.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-18T17:54:28.178 INFO:teuthology.orchestra.run.smithi072.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-18T17:54:28.180 INFO:teuthology.orchestra.run.smithi072.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-18T17:54:28.182 INFO:teuthology.orchestra.run.smithi072.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-18T17:54:28.184 INFO:teuthology.orchestra.run.smithi072.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-18T17:54:28.188 INFO:teuthology.orchestra.run.smithi179.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.2.14-100-gb4ddd52c-1focal [22.6 MB] 2023-10-18T17:54:28.188 INFO:teuthology.orchestra.run.smithi072.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-18T17:54:28.189 INFO:teuthology.orchestra.run.smithi072.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-18T17:54:28.190 INFO:teuthology.orchestra.run.smithi072.stdout:Get:40 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libjaeger amd64 16.2.14-100-gb4ddd52c-1focal [3,964 B] 2023-10-18T17:54:28.276 INFO:teuthology.orchestra.run.smithi072.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-18T17:54:28.294 INFO:teuthology.orchestra.run.smithi072.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-18T17:54:28.295 INFO:teuthology.orchestra.run.smithi072.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-18T17:54:28.309 INFO:teuthology.orchestra.run.smithi072.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-18T17:54:28.309 INFO:teuthology.orchestra.run.smithi072.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-18T17:54:28.310 INFO:teuthology.orchestra.run.smithi072.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-18T17:54:28.310 INFO:teuthology.orchestra.run.smithi072.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-18T17:54:28.311 INFO:teuthology.orchestra.run.smithi072.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-18T17:54:28.311 INFO:teuthology.orchestra.run.smithi072.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-18T17:54:28.312 INFO:teuthology.orchestra.run.smithi072.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-18T17:54:28.314 INFO:teuthology.orchestra.run.smithi072.stdout:Get:51 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.2.14-100-gb4ddd52c-1focal [3,382 kB] 2023-10-18T17:54:28.328 INFO:teuthology.orchestra.run.smithi072.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-18T17:54:28.329 INFO:teuthology.orchestra.run.smithi072.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-18T17:54:28.330 INFO:teuthology.orchestra.run.smithi072.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-18T17:54:28.337 INFO:teuthology.orchestra.run.smithi072.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-18T17:54:28.340 INFO:teuthology.orchestra.run.smithi072.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-18T17:54:28.345 INFO:teuthology.orchestra.run.smithi072.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-18T17:54:28.346 INFO:teuthology.orchestra.run.smithi072.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-18T17:54:28.348 INFO:teuthology.orchestra.run.smithi072.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-18T17:54:28.351 INFO:teuthology.orchestra.run.smithi072.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-18T17:54:28.351 INFO:teuthology.orchestra.run.smithi072.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-18T17:54:28.362 INFO:teuthology.orchestra.run.smithi072.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-18T17:54:28.362 INFO:teuthology.orchestra.run.smithi072.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-18T17:54:28.364 INFO:teuthology.orchestra.run.smithi072.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-18T17:54:28.365 INFO:teuthology.orchestra.run.smithi072.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-18T17:54:28.368 INFO:teuthology.orchestra.run.smithi072.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-18T17:54:28.384 INFO:teuthology.orchestra.run.smithi072.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-18T17:54:28.385 INFO:teuthology.orchestra.run.smithi072.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-18T17:54:28.385 INFO:teuthology.orchestra.run.smithi072.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-18T17:54:28.385 INFO:teuthology.orchestra.run.smithi072.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-18T17:54:28.387 INFO:teuthology.orchestra.run.smithi072.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-18T17:54:28.396 INFO:teuthology.orchestra.run.smithi072.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-18T17:54:28.396 INFO:teuthology.orchestra.run.smithi072.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-18T17:54:28.398 INFO:teuthology.orchestra.run.smithi072.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-18T17:54:28.620 INFO:teuthology.orchestra.run.smithi072.stdout:Get:75 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.2.14-100-gb4ddd52c-1focal [3,389 kB] 2023-10-18T17:54:28.770 INFO:teuthology.orchestra.run.smithi072.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.2.14-100-gb4ddd52c-1focal [729 kB] 2023-10-18T17:54:28.798 INFO:teuthology.orchestra.run.smithi072.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.2.14-100-gb4ddd52c-1focal [413 kB] 2023-10-18T17:54:28.816 INFO:teuthology.orchestra.run.smithi072.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.2.14-100-gb4ddd52c-1focal [22.2 kB] 2023-10-18T17:54:28.817 INFO:teuthology.orchestra.run.smithi072.stdout:Get:79 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.2.14-100-gb4ddd52c-1focal [212 kB] 2023-10-18T17:54:28.823 INFO:teuthology.orchestra.run.smithi072.stdout:Get:80 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.2.14-100-gb4ddd52c-1focal [38.4 kB] 2023-10-18T17:54:28.824 INFO:teuthology.orchestra.run.smithi072.stdout:Get:81 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.2.14-100-gb4ddd52c-1focal [387 kB] 2023-10-18T17:54:28.836 INFO:teuthology.orchestra.run.smithi072.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.2.14-100-gb4ddd52c-1focal [3,689 kB] 2023-10-18T17:54:28.999 INFO:teuthology.orchestra.run.smithi072.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.2.14-100-gb4ddd52c-1focal [116 kB] 2023-10-18T17:54:29.004 INFO:teuthology.orchestra.run.smithi072.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.2.14-100-gb4ddd52c-1focal [448 kB] 2023-10-18T17:54:29.022 INFO:teuthology.orchestra.run.smithi072.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.2.14-100-gb4ddd52c-1focal [22.6 MB] 2023-10-18T17:54:30.039 INFO:teuthology.orchestra.run.smithi072.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.2.14-100-gb4ddd52c-1focal [5,914 kB] 2023-10-18T17:54:30.336 INFO:teuthology.orchestra.run.smithi072.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.2.14-100-gb4ddd52c-1focal [204 kB] 2023-10-18T17:54:30.352 INFO:teuthology.orchestra.run.smithi072.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 16.2.14-100-gb4ddd52c-1focal [126 kB] 2023-10-18T17:54:30.358 INFO:teuthology.orchestra.run.smithi072.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.2.14-100-gb4ddd52c-1focal [1,538 kB] 2023-10-18T17:54:30.479 INFO:teuthology.orchestra.run.smithi072.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.2.14-100-gb4ddd52c-1focal [6,998 kB] 2023-10-18T17:54:30.860 INFO:teuthology.orchestra.run.smithi072.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.2.14-100-gb4ddd52c-1focal [26.2 MB] 2023-10-18T17:54:32.369 INFO:teuthology.orchestra.run.smithi179.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.2.14-100-gb4ddd52c-1focal [5,914 kB] 2023-10-18T17:54:32.550 INFO:teuthology.orchestra.run.smithi072.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.2.14-100-gb4ddd52c-1focal [4,052 B] 2023-10-18T17:54:32.551 INFO:teuthology.orchestra.run.smithi072.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.2.14-100-gb4ddd52c-1focal [835 kB] 2023-10-18T17:54:32.573 INFO:teuthology.orchestra.run.smithi072.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.2.14-100-gb4ddd52c-1focal [2,361 kB] 2023-10-18T17:54:32.678 INFO:teuthology.orchestra.run.smithi072.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.2.14-100-gb4ddd52c-1focal [76.4 kB] 2023-10-18T17:54:32.680 INFO:teuthology.orchestra.run.smithi072.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.2.14-100-gb4ddd52c-1focal [100 kB] 2023-10-18T17:54:32.684 INFO:teuthology.orchestra.run.smithi072.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.2.14-100-gb4ddd52c-1focal [2,824 kB] 2023-10-18T17:54:32.819 INFO:teuthology.orchestra.run.smithi072.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.2.14-100-gb4ddd52c-1focal [7,877 kB] 2023-10-18T17:54:33.148 INFO:teuthology.orchestra.run.smithi072.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.2.14-100-gb4ddd52c-1focal [4,168 B] 2023-10-18T17:54:33.149 INFO:teuthology.orchestra.run.smithi072.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.2.14-100-gb4ddd52c-1focal [49.3 MB] 2023-10-18T17:54:33.224 INFO:teuthology.orchestra.run.smithi179.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.2.14-100-gb4ddd52c-1focal [204 kB] 2023-10-18T17:54:33.246 INFO:teuthology.orchestra.run.smithi179.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 16.2.14-100-gb4ddd52c-1focal [126 kB] 2023-10-18T17:54:33.267 INFO:teuthology.orchestra.run.smithi179.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.2.14-100-gb4ddd52c-1focal [1,538 kB] 2023-10-18T17:54:33.492 INFO:teuthology.orchestra.run.smithi179.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.2.14-100-gb4ddd52c-1focal [6,998 kB] 2023-10-18T17:54:35.191 INFO:teuthology.orchestra.run.smithi179.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.2.14-100-gb4ddd52c-1focal [26.2 MB] 2023-10-18T17:54:36.764 INFO:teuthology.orchestra.run.smithi072.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 16.2.14-100-gb4ddd52c-1focal [237 kB] 2023-10-18T17:54:36.767 INFO:teuthology.orchestra.run.smithi072.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 16.2.14-100-gb4ddd52c-1focal [17.4 kB] 2023-10-18T17:54:36.767 INFO:teuthology.orchestra.run.smithi072.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 16.2.14-100-gb4ddd52c-1focal [13.2 kB] 2023-10-18T17:54:36.768 INFO:teuthology.orchestra.run.smithi072.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.2.14-100-gb4ddd52c-1focal [21.5 kB] 2023-10-18T17:54:36.768 INFO:teuthology.orchestra.run.smithi072.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.2.14-100-gb4ddd52c-1focal [11.3 MB] 2023-10-18T17:54:37.888 INFO:teuthology.orchestra.run.smithi072.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.2.14-100-gb4ddd52c-1focal [78.5 kB] 2023-10-18T17:54:38.790 INFO:teuthology.orchestra.run.smithi072.stdout:Fetched 174 MB in 10s (17.4 MB/s) 2023-10-18T17:54:38.952 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-18T17:54:39.024 INFO:teuthology.orchestra.run.smithi072.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107758 files and directories currently installed.) 2023-10-18T17:54:39.027 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-18T17:54:39.073 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-18T17:54:39.462 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-18T17:54:39.479 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-18T17:54:39.520 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-18T17:54:39.861 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-cmd2. 2023-10-18T17:54:39.881 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-18T17:54:39.927 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-18T17:54:40.215 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libjaeger. 2023-10-18T17:54:40.227 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../003-libjaeger_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:40.308 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libjaeger (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:40.655 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-18T17:54:40.675 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../004-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-18T17:54:40.764 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-18T17:54:41.142 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../005-librbd1_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:41.298 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking librbd1 (16.2.14-100-gb4ddd52c-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-18T17:54:42.111 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../006-librados2_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking librados2 (16.2.14-100-gb4ddd52c-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-18T17:54:42.717 INFO:teuthology.orchestra.run.smithi179.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.2.14-100-gb4ddd52c-1focal [4,052 B] 2023-10-18T17:54:42.718 INFO:teuthology.orchestra.run.smithi179.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.2.14-100-gb4ddd52c-1focal [835 kB] 2023-10-18T17:54:42.800 INFO:teuthology.orchestra.run.smithi179.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.2.14-100-gb4ddd52c-1focal [2,361 kB] 2023-10-18T17:54:43.319 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libcephfs2. 2023-10-18T17:54:43.343 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../007-libcephfs2_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:43.369 INFO:teuthology.orchestra.run.smithi179.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.2.14-100-gb4ddd52c-1focal [76.4 kB] 2023-10-18T17:54:43.370 INFO:teuthology.orchestra.run.smithi179.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.2.14-100-gb4ddd52c-1focal [100 kB] 2023-10-18T17:54:43.386 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libcephfs2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:43.418 INFO:teuthology.orchestra.run.smithi179.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.2.14-100-gb4ddd52c-1focal [2,824 kB] 2023-10-18T17:54:43.767 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-rados. 2023-10-18T17:54:43.787 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../008-python3-rados_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:43.825 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-rados (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:44.071 INFO:teuthology.orchestra.run.smithi179.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.2.14-100-gb4ddd52c-1focal [7,877 kB] 2023-10-18T17:54:44.206 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-18T17:54:44.224 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:54:44.297 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-ceph-argparse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:44.637 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-cephfs. 2023-10-18T17:54:44.657 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../010-python3-cephfs_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:44.703 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-cephfs (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:45.077 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-18T17:54:45.098 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../011-python3-ceph-common_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:54:45.143 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-ceph-common (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:45.541 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-prettytable. 2023-10-18T17:54:45.561 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-5_all.deb ... 2023-10-18T17:54:45.607 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-18T17:54:45.889 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-rbd. 2023-10-18T17:54:45.907 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../013-python3-rbd_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:45.947 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-rbd (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:46.336 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-18T17:54:46.353 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../014-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-18T17:54:46.403 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-18T17:54:46.686 INFO:teuthology.orchestra.run.smithi179.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.2.14-100-gb4ddd52c-1focal [4,168 B] 2023-10-18T17:54:46.721 INFO:teuthology.orchestra.run.smithi179.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.2.14-100-gb4ddd52c-1focal [49.3 MB] 2023-10-18T17:54:46.769 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-18T17:54:46.789 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../015-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-18T17:54:46.835 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-18T17:54:47.251 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-18T17:54:47.268 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../016-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-18T17:54:47.309 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-18T17:54:47.682 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package librgw2. 2023-10-18T17:54:47.701 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../017-librgw2_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:47.741 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking librgw2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:48.496 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-rgw. 2023-10-18T17:54:48.513 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../018-python3-rgw_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:48.554 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-rgw (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:48.910 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libleveldb1d:amd64. 2023-10-18T17:54:48.932 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../019-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2023-10-18T17:54:48.977 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2023-10-18T17:54:49.349 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package liboath0. 2023-10-18T17:54:49.367 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../020-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-18T17:54:49.407 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-18T17:54:49.739 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libradosstriper1. 2023-10-18T17:54:49.759 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../021-libradosstriper1_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:49.797 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libradosstriper1 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:50.321 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-common. 2023-10-18T17:54:50.343 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../022-ceph-common_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:50.388 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-common (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:52.750 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-base. 2023-10-18T17:54:52.769 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../023-ceph-base_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:54:52.828 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-base (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:54.394 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-cryptography. 2023-10-18T17:54:54.413 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../024-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-18T17:54:54.460 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-18T17:54:54.877 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-openssl. 2023-10-18T17:54:54.895 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../025-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-18T17:54:54.935 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-18T17:54:55.215 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-18T17:54:55.232 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:54:55.274 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-mgr-modules-core (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:54:55.683 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-18T17:54:55.703 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-18T17:54:55.743 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-18T17:54:56.082 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-18T17:54:56.102 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-18T17:54:56.148 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-18T17:54:56.548 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-jwt. 2023-10-18T17:54:56.565 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../029-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-18T17:54:56.606 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-18T17:54:56.955 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-logutils. 2023-10-18T17:54:56.976 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-10-18T17:54:57.013 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-18T17:54:57.385 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-18T17:54:57.405 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-18T17:54:57.452 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-18T17:54:57.809 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-mako. 2023-10-18T17:54:57.832 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-18T17:54:57.875 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-18T17:54:58.239 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-18T17:54:58.257 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-18T17:54:58.299 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-18T17:54:58.671 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-18T17:54:58.691 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-18T17:54:58.729 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-18T17:55:00.420 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-webob. 2023-10-18T17:55:00.440 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-18T17:55:00.486 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-18T17:55:00.875 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-18T17:55:00.892 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-18T17:55:00.933 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-18T17:55:01.315 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-bs4. 2023-10-18T17:55:01.334 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-10-18T17:55:01.373 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-18T17:55:01.764 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-waitress. 2023-10-18T17:55:01.784 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-18T17:55:01.838 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-18T17:55:02.237 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-tempita. 2023-10-18T17:55:02.255 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-10-18T17:55:02.296 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-18T17:55:02.627 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-paste. 2023-10-18T17:55:02.644 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-18T17:55:02.685 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-18T17:55:03.084 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-18T17:55:03.102 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-18T17:55:03.142 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-18T17:55:03.473 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-18T17:55:03.491 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-18T17:55:03.539 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-18T17:55:03.904 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-webtest. 2023-10-18T17:55:03.925 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-10-18T17:55:03.971 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-18T17:55:04.361 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-pecan. 2023-10-18T17:55:04.382 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-18T17:55:04.428 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-18T17:55:04.750 INFO:teuthology.orchestra.run.smithi179.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 16.2.14-100-gb4ddd52c-1focal [237 kB] 2023-10-18T17:55:04.755 INFO:teuthology.orchestra.run.smithi179.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 16.2.14-100-gb4ddd52c-1focal [17.4 kB] 2023-10-18T17:55:04.756 INFO:teuthology.orchestra.run.smithi179.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 16.2.14-100-gb4ddd52c-1focal [13.2 kB] 2023-10-18T17:55:04.757 INFO:teuthology.orchestra.run.smithi179.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.2.14-100-gb4ddd52c-1focal [21.5 kB] 2023-10-18T17:55:04.757 INFO:teuthology.orchestra.run.smithi179.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.2.14-100-gb4ddd52c-1focal [11.3 MB] 2023-10-18T17:55:04.830 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-18T17:55:04.850 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-18T17:55:04.896 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-18T17:55:05.294 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-18T17:55:05.313 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../046-libsqlite3-mod-ceph_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:05.362 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:05.743 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-mgr. 2023-10-18T17:55:05.743 INFO:teuthology.orchestra.run.smithi179.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-pacific-20231016.091648/b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.2.14-100-gb4ddd52c-1focal [78.5 kB] 2023-10-18T17:55:05.764 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../047-ceph-mgr_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:05.809 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-mgr (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:06.346 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 174 MB in 39s (4,407 kB/s) 2023-10-18T17:55:06.383 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-mon. 2023-10-18T17:55:06.400 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../048-ceph-mon_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:06.441 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-mon (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:06.486 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-18T17:55:06.533 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 105332 files and directories currently installed.) 2023-10-18T17:55:06.537 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-18T17:55:06.576 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-18T17:55:06.845 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-18T17:55:06.858 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-18T17:55:06.895 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-18T17:55:07.206 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cmd2. 2023-10-18T17:55:07.222 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-18T17:55:07.264 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-18T17:55:07.517 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjaeger. 2023-10-18T17:55:07.533 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../003-libjaeger_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:07.575 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjaeger (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:07.658 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-osd. 2023-10-18T17:55:07.678 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../049-ceph-osd_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:07.724 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-osd (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:07.878 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-18T17:55:07.890 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../004-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-18T17:55:07.953 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-18T17:55:08.273 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../005-librbd1_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:08.432 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librbd1 (16.2.14-100-gb4ddd52c-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-18T17:55:09.135 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../006-librados2_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:09.237 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librados2 (16.2.14-100-gb4ddd52c-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-18T17:55:09.924 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libcephfs2. 2023-10-18T17:55:09.937 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../007-libcephfs2_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:09.982 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libcephfs2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:10.303 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph. 2023-10-18T17:55:10.323 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../050-ceph_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:10.335 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rados. 2023-10-18T17:55:10.350 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../008-python3-rados_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:10.361 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:10.385 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rados (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:10.704 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-18T17:55:10.720 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:10.724 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-fuse. 2023-10-18T17:55:10.743 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../051-ceph-fuse_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:10.762 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-ceph-argparse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:10.782 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-fuse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:11.090 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cephfs. 2023-10-18T17:55:11.104 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../010-python3-cephfs_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:11.148 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cephfs (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:11.265 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-mds. 2023-10-18T17:55:11.283 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../052-ceph-mds_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:11.323 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-mds (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:11.476 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-18T17:55:11.491 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../011-python3-ceph-common_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:11.568 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-ceph-common (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:11.846 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-prettytable. 2023-10-18T17:55:11.862 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-5_all.deb ... 2023-10-18T17:55:11.895 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-18T17:55:11.921 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package cephadm. 2023-10-18T17:55:11.941 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../053-cephadm_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:11.988 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking cephadm (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:12.123 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rbd. 2023-10-18T17:55:12.136 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../013-python3-rbd_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:12.164 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rbd (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:12.343 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-jinja2. 2023-10-18T17:55:12.363 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../054-python3-jinja2_2.10.1-2_all.deb ... 2023-10-18T17:55:12.443 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-18T17:55:12.509 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-18T17:55:12.521 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../014-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-18T17:55:12.559 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-18T17:55:12.799 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-18T17:55:12.812 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../055-ceph-mgr-cephadm_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:12.857 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-mgr-cephadm (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:12.929 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-18T17:55:12.942 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../015-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-18T17:55:12.978 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-18T17:55:13.198 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-18T17:55:13.211 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../056-python3-repoze.lru_0.7-2_all.deb ... 2023-10-18T17:55:13.256 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-18T17:55:13.273 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-18T17:55:13.289 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../016-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-18T17:55:13.323 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-18T17:55:13.611 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libjs-underscore. 2023-10-18T17:55:13.631 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../057-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-18T17:55:13.669 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-18T17:55:13.676 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package librgw2. 2023-10-18T17:55:13.691 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../017-librgw2_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:13.743 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking librgw2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:13.983 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-18T17:55:14.003 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../058-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-18T17:55:14.050 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-18T17:55:14.379 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rgw. 2023-10-18T17:55:14.392 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../018-python3-rgw_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:14.421 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-routes. 2023-10-18T17:55:14.436 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rgw (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:14.442 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../059-python3-routes_2.4.1-2_all.deb ... 2023-10-18T17:55:14.488 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-18T17:55:14.823 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libleveldb1d:amd64. 2023-10-18T17:55:14.836 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../019-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2023-10-18T17:55:14.853 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-18T17:55:14.871 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:14.881 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2023-10-18T17:55:14.919 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-mgr-dashboard (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:15.176 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liboath0. 2023-10-18T17:55:15.189 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../020-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-18T17:55:15.234 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-18T17:55:15.553 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libradosstriper1. 2023-10-18T17:55:15.569 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../021-libradosstriper1_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:15.603 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libradosstriper1 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:16.048 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-common. 2023-10-18T17:55:16.061 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../022-ceph-common_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:16.087 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-decorator. 2023-10-18T17:55:16.108 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../061-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-18T17:55:16.154 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-18T17:55:16.240 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-common (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:16.510 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-18T17:55:16.530 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../062-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-18T17:55:16.577 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-18T17:55:16.940 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-scipy. 2023-10-18T17:55:16.960 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../063-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-18T17:55:16.999 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-18T17:55:18.404 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-base. 2023-10-18T17:55:18.419 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../023-ceph-base_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:18.472 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-base (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:18.542 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-18T17:55:18.563 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-18T17:55:18.610 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-18T17:55:19.225 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-joblib. 2023-10-18T17:55:19.243 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../065-python3-joblib_0.14.0-3_all.deb ... 2023-10-18T17:55:19.291 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-18T17:55:19.433 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cryptography. 2023-10-18T17:55:19.446 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../024-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-18T17:55:19.483 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-18T17:55:19.741 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-sklearn. 2023-10-18T17:55:19.762 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../066-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-18T17:55:19.807 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-18T17:55:19.845 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-openssl. 2023-10-18T17:55:19.860 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../025-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-18T17:55:19.912 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-18T17:55:20.164 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-18T17:55:20.179 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../026-ceph-mgr-modules-core_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:20.223 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-modules-core (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:20.591 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-18T17:55:20.594 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-18T17:55:20.608 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../027-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-18T17:55:20.609 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:20.644 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-18T17:55:20.660 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:20.913 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-18T17:55:20.928 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-18T17:55:20.971 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-18T17:55:21.300 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-jwt. 2023-10-18T17:55:21.315 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../029-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-18T17:55:21.350 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-18T17:55:21.881 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-logutils. 2023-10-18T17:55:21.897 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-10-18T17:55:21.953 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-18T17:55:22.150 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-cachetools. 2023-10-18T17:55:22.169 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../068-python3-cachetools_4.0.0-1_all.deb ... 2023-10-18T17:55:22.216 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-18T17:55:22.239 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-18T17:55:22.255 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-18T17:55:22.297 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-18T17:55:22.581 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-18T17:55:22.601 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../069-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-18T17:55:22.625 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-mako. 2023-10-18T17:55:22.638 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-18T17:55:22.638 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-18T17:55:22.675 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-18T17:55:22.970 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-18T17:55:22.982 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-18T17:55:23.005 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-18T17:55:23.019 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-18T17:55:23.028 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../070-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-18T17:55:23.071 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-18T17:55:23.322 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-18T17:55:23.337 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-18T17:55:23.381 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-18T17:55:23.451 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-rsa. 2023-10-18T17:55:23.473 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../071-python3-rsa_4.0-3_all.deb ... 2023-10-18T17:55:23.520 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-18T17:55:23.642 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-webob. 2023-10-18T17:55:23.654 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-18T17:55:23.683 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-18T17:55:23.850 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-google-auth. 2023-10-18T17:55:23.871 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../072-python3-google-auth_1.5.1-2_all.deb ... 2023-10-18T17:55:23.916 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-18T17:55:24.011 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-18T17:55:24.027 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-18T17:55:24.069 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-18T17:55:24.283 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-blinker. 2023-10-18T17:55:24.301 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../073-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-18T17:55:24.357 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-18T17:55:24.372 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-bs4. 2023-10-18T17:55:24.385 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-10-18T17:55:24.422 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-18T17:55:24.688 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-18T17:55:24.708 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../074-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-18T17:55:24.754 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-18T17:55:24.791 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-waitress. 2023-10-18T17:55:24.805 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-18T17:55:24.853 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-18T17:55:25.137 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-18T17:55:25.158 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../075-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-18T17:55:25.204 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-18T17:55:25.228 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-tempita. 2023-10-18T17:55:25.243 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-10-18T17:55:25.286 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-18T17:55:25.589 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-paste. 2023-10-18T17:55:25.602 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-18T17:55:25.639 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-18T17:55:25.892 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-websocket. 2023-10-18T17:55:25.913 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../076-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-18T17:55:25.950 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-18T17:55:25.984 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-18T17:55:25.998 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-18T17:55:26.042 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-18T17:55:26.299 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-18T17:55:26.325 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../077-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-18T17:55:26.387 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-18T17:55:26.402 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-18T17:55:26.407 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-18T17:55:26.445 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-18T17:55:26.731 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-webtest. 2023-10-18T17:55:26.747 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-10-18T17:55:26.789 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-18T17:55:27.034 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-18T17:55:27.053 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:27.100 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:27.125 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pecan. 2023-10-18T17:55:27.140 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-18T17:55:27.184 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-18T17:55:27.474 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-18T17:55:27.493 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../079-libonig5_6.9.4-1_amd64.deb ... 2023-10-18T17:55:27.522 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-18T17:55:27.530 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-18T17:55:27.536 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-18T17:55:27.580 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-18T17:55:27.920 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-18T17:55:27.941 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../080-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-18T17:55:27.941 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-18T17:55:27.956 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../046-libsqlite3-mod-ceph_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:27.978 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-18T17:55:28.000 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:28.361 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr. 2023-10-18T17:55:28.377 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../047-ceph-mgr_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:28.419 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:28.432 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package jq. 2023-10-18T17:55:28.452 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../081-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-18T17:55:28.498 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-18T17:55:28.797 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package socat. 2023-10-18T17:55:28.820 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../082-socat_1.7.3.3-2_amd64.deb ... 2023-10-18T17:55:28.863 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-18T17:55:28.948 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mon. 2023-10-18T17:55:28.963 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../048-ceph-mon_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:29.006 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mon (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:29.212 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package xmlstarlet. 2023-10-18T17:55:29.233 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../083-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-18T17:55:29.270 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-18T17:55:29.689 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ceph-test. 2023-10-18T17:55:29.710 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../084-ceph-test_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:29.756 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ceph-test (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:30.051 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-osd. 2023-10-18T17:55:30.065 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../049-ceph-osd_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:30.109 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-osd (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:32.790 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph. 2023-10-18T17:55:32.806 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../050-ceph_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:32.848 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:33.167 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-fuse. 2023-10-18T17:55:33.181 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../051-ceph-fuse_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:33.225 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-fuse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:33.679 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mds. 2023-10-18T17:55:33.693 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../052-ceph-mds_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:33.737 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mds (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:34.014 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-18T17:55:34.028 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../085-cephfs-mirror_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:34.074 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking cephfs-mirror (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:34.298 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephadm. 2023-10-18T17:55:34.311 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../053-cephadm_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:34.348 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephadm (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:34.464 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-colorama. 2023-10-18T17:55:34.484 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../086-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-18T17:55:34.522 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-18T17:55:34.650 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-jinja2. 2023-10-18T17:55:34.666 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../054-python3-jinja2_2.10.1-2_all.deb ... 2023-10-18T17:55:34.725 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-18T17:55:34.812 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package cephfs-shell. 2023-10-18T17:55:34.832 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../087-cephfs-shell_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:34.912 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking cephfs-shell (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:35.070 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-18T17:55:35.084 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../055-ceph-mgr-cephadm_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:35.128 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-cephadm (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:35.192 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package cephfs-top. 2023-10-18T17:55:35.211 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../088-cephfs-top_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:35.258 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking cephfs-top (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:35.498 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-18T17:55:35.512 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../056-python3-repoze.lru_0.7-2_all.deb ... 2023-10-18T17:55:35.556 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-18T17:55:35.598 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-18T17:55:35.623 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../089-libcephfs-dev_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:35.664 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libcephfs-dev (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:35.792 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjs-underscore. 2023-10-18T17:55:35.806 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../057-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-18T17:55:35.850 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-18T17:55:36.122 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package nvme-cli. 2023-10-18T17:55:36.140 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-18T17:55:36.144 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-18T17:55:36.159 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../058-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-18T17:55:36.203 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-18T17:55:36.217 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-18T17:55:36.522 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-routes. 2023-10-18T17:55:36.536 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../059-python3-routes_2.4.1-2_all.deb ... 2023-10-18T17:55:36.580 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-18T17:55:36.618 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-18T17:55:36.638 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../091-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-18T17:55:36.676 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-18T17:55:36.941 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-18T17:55:36.956 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:36.999 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-dashboard (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:37.058 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-attr. 2023-10-18T17:55:37.077 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../092-python3-attr_19.3.0-2_all.deb ... 2023-10-18T17:55:37.116 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-18T17:55:37.471 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-webencodings. 2023-10-18T17:55:37.493 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../093-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-18T17:55:37.537 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-18T17:55:37.919 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-html5lib. 2023-10-18T17:55:37.941 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../094-python3-html5lib_1.0.1-2_all.deb ... 2023-10-18T17:55:37.977 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-18T17:55:38.066 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-decorator. 2023-10-18T17:55:38.083 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../061-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-18T17:55:38.124 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-18T17:55:38.350 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-18T17:55:38.368 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../095-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-18T17:55:38.407 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-18T17:55:38.424 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-18T17:55:38.439 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../062-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-18T17:55:38.482 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-18T17:55:38.827 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-scipy. 2023-10-18T17:55:38.841 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../063-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-18T17:55:38.885 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-18T17:55:38.931 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-packaging. 2023-10-18T17:55:38.949 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../096-python3-packaging_20.3-1_all.deb ... 2023-10-18T17:55:38.989 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-18T17:55:39.338 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-pastescript. 2023-10-18T17:55:39.359 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../097-python3-pastescript_2.0.2-4_all.deb ... 2023-10-18T17:55:39.404 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-18T17:55:39.760 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-pluggy. 2023-10-18T17:55:39.782 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../098-python3-pluggy_0.13.0-2_all.deb ... 2023-10-18T17:55:39.848 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-18T17:55:40.127 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-18T17:55:40.141 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-18T17:55:40.185 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-18T17:55:40.193 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-psutil. 2023-10-18T17:55:40.213 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../099-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-18T17:55:40.258 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-18T17:55:40.657 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-py. 2023-10-18T17:55:40.677 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../100-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-18T17:55:40.723 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-18T17:55:40.764 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-joblib. 2023-10-18T17:55:40.780 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../065-python3-joblib_0.14.0-3_all.deb ... 2023-10-18T17:55:40.822 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-18T17:55:41.104 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-18T17:55:41.125 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-18T17:55:41.171 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-18T17:55:41.210 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-sklearn. 2023-10-18T17:55:41.225 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../066-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-18T17:55:41.267 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-18T17:55:41.511 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-pytest. 2023-10-18T17:55:41.531 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../102-python3-pytest_4.6.9-1_all.deb ... 2023-10-18T17:55:41.568 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-18T17:55:41.960 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package python3-simplejson. 2023-10-18T17:55:41.980 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../103-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-18T17:55:42.013 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-18T17:55:42.026 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-18T17:55:42.028 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:42.071 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:42.491 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package radosgw. 2023-10-18T17:55:42.509 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../104-radosgw_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:42.548 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking radosgw (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:43.174 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-cachetools. 2023-10-18T17:55:43.187 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../068-python3-cachetools_4.0.0-1_all.deb ... 2023-10-18T17:55:43.232 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-18T17:55:43.536 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-18T17:55:43.549 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../069-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-18T17:55:43.594 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-18T17:55:43.900 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package rbd-fuse. 2023-10-18T17:55:43.917 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../105-rbd-fuse_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:43.939 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-18T17:55:43.955 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../070-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-18T17:55:43.966 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking rbd-fuse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:43.997 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-18T17:55:44.344 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-18T17:55:44.358 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-rsa. 2023-10-18T17:55:44.371 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../071-python3-rsa_4.0-3_all.deb ... 2023-10-18T17:55:44.408 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-18T17:55:44.695 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-google-auth. 2023-10-18T17:55:44.711 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../072-python3-google-auth_1.5.1-2_all.deb ... 2023-10-18T17:55:44.718 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2023-10-18T17:55:44.753 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-18T17:55:44.896 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-18T17:55:45.023 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-18T17:55:45.098 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-blinker. 2023-10-18T17:55:45.113 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../073-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-18T17:55:45.156 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-18T17:55:45.251 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up cephadm (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:45.443 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-18T17:55:45.459 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../074-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-18T17:55:45.500 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-18T17:55:45.846 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-18T17:55:45.860 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../075-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-18T17:55:45.904 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-18T17:55:46.023 INFO:teuthology.orchestra.run.smithi072.stdout:Adding system user cephadm....done 2023-10-18T17:55:46.207 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-websocket. 2023-10-18T17:55:46.217 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-18T17:55:46.221 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../076-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-18T17:55:46.281 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-18T17:55:46.593 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-18T17:55:46.608 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../077-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-18T17:55:46.651 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-18T17:55:46.654 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-18T17:55:47.006 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-18T17:55:47.125 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-18T17:55:47.231 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-18T17:55:47.245 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:47.289 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:47.529 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-18T17:55:47.642 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-18T17:55:47.656 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../079-libonig5_6.9.4-1_amd64.deb ... 2023-10-18T17:55:47.700 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-18T17:55:48.016 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-18T17:55:48.053 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-18T17:55:48.068 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../080-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-18T17:55:48.111 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-18T17:55:48.377 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-18T17:55:48.482 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package jq. 2023-10-18T17:55:48.497 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../081-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-18T17:55:48.532 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-18T17:55:48.739 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-18T17:55:48.751 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package socat. 2023-10-18T17:55:48.767 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../082-socat_1.7.3.3-2_amd64.deb ... 2023-10-18T17:55:48.801 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-18T17:55:49.124 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-18T17:55:49.140 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package xmlstarlet. 2023-10-18T17:55:49.154 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../083-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-18T17:55:49.198 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-18T17:55:49.468 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-18T17:55:49.578 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ceph-test. 2023-10-18T17:55:49.593 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../084-ceph-test_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:49.636 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ceph-test (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:49.862 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-ceph-argparse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:50.215 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-18T17:55:50.342 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-18T17:55:50.702 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-18T17:55:51.108 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-18T17:55:51.459 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-18T17:55:51.936 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-18T17:55:52.288 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-18T17:55:52.717 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-18T17:55:53.113 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libjaeger (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:53.248 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-18T17:55:53.658 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-18T17:55:53.973 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-18T17:55:53.988 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../085-cephfs-mirror_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:54.019 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-18T17:55:54.048 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephfs-mirror (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:54.360 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-colorama. 2023-10-18T17:55:54.374 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-18T17:55:54.375 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../086-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-18T17:55:54.409 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-18T17:55:54.500 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-18T17:55:54.604 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephfs-shell. 2023-10-18T17:55:54.617 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../087-cephfs-shell_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:54.695 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephfs-shell (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:54.846 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-18T17:55:54.956 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package cephfs-top. 2023-10-18T17:55:54.972 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../088-cephfs-top_16.2.14-100-gb4ddd52c-1focal_all.deb ... 2023-10-18T17:55:55.014 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking cephfs-top (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:55.267 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-18T17:55:55.284 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../089-libcephfs-dev_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:55:55.332 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-18T17:55:55.342 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libcephfs-dev (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:55.707 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-18T17:55:55.737 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package nvme-cli. 2023-10-18T17:55:55.750 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-18T17:55:55.813 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-18T17:55:56.209 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-18T17:55:56.222 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../091-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-18T17:55:56.267 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-18T17:55:56.376 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-18T17:55:56.611 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-attr. 2023-10-18T17:55:56.627 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../092-python3-attr_19.3.0-2_all.deb ... 2023-10-18T17:55:56.669 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-18T17:55:56.823 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-18T17:55:56.966 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-ceph-common (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:55:56.980 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-webencodings. 2023-10-18T17:55:56.993 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../093-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-18T17:55:57.038 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-18T17:55:57.343 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-18T17:55:57.375 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-html5lib. 2023-10-18T17:55:57.389 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../094-python3-html5lib_1.0.1-2_all.deb ... 2023-10-18T17:55:57.433 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-18T17:55:57.769 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-18T17:55:57.784 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../095-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-18T17:55:57.805 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-18T17:55:57.827 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-18T17:55:57.949 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-18T17:55:58.076 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-18T17:55:58.290 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-packaging. 2023-10-18T17:55:58.305 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../096-python3-packaging_20.3-1_all.deb ... 2023-10-18T17:55:58.348 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-18T17:55:58.436 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-18T17:55:58.684 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pastescript. 2023-10-18T17:55:58.699 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../097-python3-pastescript_2.0.2-4_all.deb ... 2023-10-18T17:55:58.742 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-18T17:55:58.824 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-18T17:55:58.950 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-18T17:55:59.062 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pluggy. 2023-10-18T17:55:59.077 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../098-python3-pluggy_0.13.0-2_all.deb ... 2023-10-18T17:55:59.154 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-18T17:55:59.353 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-18T17:55:59.465 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-psutil. 2023-10-18T17:55:59.480 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../099-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-18T17:55:59.523 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-18T17:55:59.772 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-18T17:55:59.902 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-py. 2023-10-18T17:55:59.916 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../100-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-18T17:55:59.960 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-18T17:56:00.134 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-18T17:56:00.246 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-18T17:56:00.261 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-18T17:56:00.303 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-18T17:56:00.304 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-18T17:56:00.604 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-18T17:56:00.649 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-pytest. 2023-10-18T17:56:00.662 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../102-python3-pytest_4.6.9-1_all.deb ... 2023-10-18T17:56:00.707 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-18T17:56:00.965 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-18T17:56:01.081 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package python3-simplejson. 2023-10-18T17:56:01.094 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../103-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-18T17:56:01.131 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-18T17:56:01.352 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-18T17:56:01.551 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package radosgw. 2023-10-18T17:56:01.565 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../104-radosgw_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:56:01.609 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking radosgw (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:01.780 INFO:teuthology.orchestra.run.smithi072.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-18T17:56:02.098 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-18T17:56:02.225 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-18T17:56:02.620 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-18T17:56:02.812 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package rbd-fuse. 2023-10-18T17:56:02.826 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../105-rbd-fuse_16.2.14-100-gb4ddd52c-1focal_amd64.deb ... 2023-10-18T17:56:02.879 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking rbd-fuse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:03.083 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-18T17:56:03.210 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-18T17:56:03.223 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-18T17:56:03.342 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-18T17:56:03.525 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2023-10-18T17:56:03.677 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-18T17:56:03.764 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-18T17:56:03.803 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-18T17:56:03.996 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephadm (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:04.116 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up librados2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:04.251 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-18T17:56:04.670 INFO:teuthology.orchestra.run.smithi179.stdout:Adding system user cephadm....done 2023-10-18T17:56:04.832 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-18T17:56:05.209 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-18T17:56:05.512 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-18T17:56:05.678 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-18T17:56:06.015 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-18T17:56:06.037 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up librgw2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:06.164 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libsqlite3-mod-ceph (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:06.292 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-18T17:56:06.419 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-18T17:56:06.442 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-18T17:56:06.718 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-18T17:56:06.872 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-18T17:56:07.035 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-18T17:56:07.249 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-18T17:56:07.320 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-18T17:56:07.596 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-18T17:56:07.604 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-18T17:56:07.905 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-ceph-argparse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:08.043 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libcephfs2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:08.171 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-18T17:56:08.215 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-18T17:56:08.341 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-18T17:56:08.566 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libradosstriper1 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:08.617 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-18T17:56:08.735 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-18T17:56:08.910 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-18T17:56:09.145 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-18T17:56:09.186 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-18T17:56:09.570 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-18T17:56:09.855 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-18T17:56:10.172 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-18T17:56:10.400 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up librbd1 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:10.465 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjaeger (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:10.544 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-18T17:56:10.591 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-18T17:56:10.712 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-18T17:56:10.839 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-18T17:56:10.951 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-18T17:56:11.201 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-18T17:56:11.243 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-18T17:56:11.569 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-18T17:56:11.603 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-mgr-modules-core (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:11.695 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-18T17:56:11.772 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-fuse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:11.980 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-18T17:56:12.037 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-18T17:56:12.037 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-18T17:56:12.374 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-18T17:56:12.683 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-18T17:56:12.876 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libcephfs-dev (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:13.003 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-18T17:56:13.053 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-18T17:56:13.355 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-18T17:56:13.431 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-18T17:56:13.556 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-ceph-common (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:13.742 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-18T17:56:13.900 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-18T17:56:14.180 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-rados (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:14.251 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-18T17:56:14.307 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up cephfs-top (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:14.377 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-18T17:56:14.536 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-18T17:56:14.548 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-rbd (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:14.716 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up rbd-fuse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:14.832 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-18T17:56:14.844 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-18T17:56:15.175 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-18T17:56:15.204 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-18T17:56:15.301 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-18T17:56:15.628 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-18T17:56:15.633 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-rgw (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:15.760 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-cephfs (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:15.912 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-18T17:56:16.146 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up cephfs-shell (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:16.205 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-18T17:56:16.274 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-18T17:56:16.331 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-18T17:56:16.643 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-18T17:56:16.944 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-18T17:56:17.270 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-18T17:56:17.442 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-common (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:17.662 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-18T17:56:17.922 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-18T17:56:18.048 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-18T17:56:18.069 INFO:teuthology.orchestra.run.smithi072.stdout:Adding group ceph....done 2023-10-18T17:56:18.383 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-18T17:56:18.527 INFO:teuthology.orchestra.run.smithi072.stdout:Adding system user ceph....done 2023-10-18T17:56:18.742 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-18T17:56:18.788 INFO:teuthology.orchestra.run.smithi072.stdout:Setting system user ceph properties....done 2023-10-18T17:56:18.825 INFO:teuthology.orchestra.run.smithi072.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-18T17:56:18.868 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-18T17:56:19.011 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-18T17:56:19.022 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-18T17:56:19.320 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-18T17:56:19.554 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-18T17:56:19.617 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librados2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:19.748 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-18T17:56:20.463 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up cephfs-mirror (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:20.758 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-18T17:56:20.759 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-18T17:56:21.187 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librgw2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:21.313 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libsqlite3-mod-ceph (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:21.439 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-18T17:56:21.598 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-18T17:56:21.615 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-test (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:21.742 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up radosgw (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:21.958 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-18T17:56:22.111 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-18T17:56:22.111 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-18T17:56:22.251 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-18T17:56:22.535 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-18T17:56:22.861 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libcephfs2 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:22.987 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-18T17:56:23.306 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libradosstriper1 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:23.349 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-base (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:23.432 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-18T17:56:23.800 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-18T17:56:23.843 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-18T17:56:24.744 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-mds (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:24.822 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up librbd1 (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:24.948 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-18T17:56:25.014 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-18T17:56:25.014 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-18T17:56:25.074 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-18T17:56:25.208 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-18T17:56:25.526 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-18T17:56:25.827 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-modules-core (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:25.881 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-mgr (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:25.953 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-fuse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:26.155 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-18T17:56:26.155 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-18T17:56:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-18T17:56:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-18T17:56:26.688 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libcephfs-dev (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:26.814 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-18T17:56:27.098 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-18T17:56:27.285 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-osd (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:27.418 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-18T17:56:27.746 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rados (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:27.855 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephfs-top (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:28.078 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-18T17:56:28.079 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-18T17:56:28.099 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rbd (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:28.225 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up rbd-fuse (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:28.360 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-18T17:56:28.869 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-18T17:56:28.930 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-mgr-k8sevents (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:29.066 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:29.179 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-rgw (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:29.218 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-mon (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:29.330 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-cephfs (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:29.483 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-18T17:56:29.483 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-18T17:56:29.632 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephfs-shell (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:29.757 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-18T17:56:30.346 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-mgr-cephadm (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:30.531 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:30.659 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ceph-mgr-dashboard (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:30.736 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-common (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:30.812 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-18T17:56:31.272 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-18T17:56:31.281 INFO:teuthology.orchestra.run.smithi179.stdout:Adding group ceph....done 2023-10-18T17:56:31.644 INFO:teuthology.orchestra.run.smithi179.stdout:Adding system user ceph....done 2023-10-18T17:56:31.870 INFO:teuthology.orchestra.run.smithi179.stdout:Setting system user ceph properties....done 2023-10-18T17:56:31.895 INFO:teuthology.orchestra.run.smithi179.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-18T17:56:32.042 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-18T17:56:32.366 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-18T17:56:32.816 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up cephfs-mirror (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:33.040 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-18T17:56:33.040 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-18T17:56:33.492 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-test (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:33.618 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up radosgw (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:33.923 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-18T17:56:33.923 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-18T17:56:34.570 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-base (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:34.952 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-18T17:56:35.406 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mds (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:35.633 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-18T17:56:35.634 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-18T17:56:36.073 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:36.298 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-18T17:56:36.299 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-18T17:56:36.345 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-18T17:56:36.750 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-osd (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:37.036 INFO:teuthology.orchestra.run.smithi072.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-18T17:56:37.042 DEBUG:teuthology.orchestra.run.smithi072:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-18T17:56:37.170 INFO:teuthology.orchestra.run.smithi072.stdout:Reading package lists... 2023-10-18T17:56:37.272 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-18T17:56:37.273 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-18T17:56:37.373 INFO:teuthology.orchestra.run.smithi072.stdout:Building dependency tree... 2023-10-18T17:56:37.374 INFO:teuthology.orchestra.run.smithi072.stdout:Reading state information... 2023-10-18T17:56:37.578 INFO:teuthology.orchestra.run.smithi072.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-18T17:56:37.579 INFO:teuthology.orchestra.run.smithi072.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-18T17:56:37.579 INFO:teuthology.orchestra.run.smithi072.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-18T17:56:37.579 INFO:teuthology.orchestra.run.smithi072.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-18T17:56:37.579 INFO:teuthology.orchestra.run.smithi072.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-18T17:56:37.580 INFO:teuthology.orchestra.run.smithi072.stdout:The following packages were automatically installed and are no longer required: 2023-10-18T17:56:37.580 INFO:teuthology.orchestra.run.smithi072.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-18T17:56:37.581 INFO:teuthology.orchestra.run.smithi072.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-18T17:56:37.581 INFO:teuthology.orchestra.run.smithi072.stdout:The following additional packages will be installed: 2023-10-18T17:56:37.581 INFO:teuthology.orchestra.run.smithi072.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-18T17:56:37.581 INFO:teuthology.orchestra.run.smithi072.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-18T17:56:37.582 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-18T17:56:37.582 INFO:teuthology.orchestra.run.smithi072.stdout: usb-modeswitch-data wpasupplicant 2023-10-18T17:56:37.582 INFO:teuthology.orchestra.run.smithi072.stdout:Suggested packages: 2023-10-18T17:56:37.582 INFO:teuthology.orchestra.run.smithi072.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-18T17:56:37.583 INFO:teuthology.orchestra.run.smithi072.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-18T17:56:37.670 INFO:teuthology.orchestra.run.smithi072.stdout:The following NEW packages will be installed: 2023-10-18T17:56:37.671 INFO:teuthology.orchestra.run.smithi072.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-18T17:56:37.671 INFO:teuthology.orchestra.run.smithi072.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-18T17:56:37.672 INFO:teuthology.orchestra.run.smithi072.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-18T17:56:37.672 INFO:teuthology.orchestra.run.smithi072.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-18T17:56:37.673 INFO:teuthology.orchestra.run.smithi072.stdout:The following packages will be upgraded: 2023-10-18T17:56:37.675 INFO:teuthology.orchestra.run.smithi072.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-18T17:56:37.710 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-k8sevents (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:37.744 INFO:teuthology.orchestra.run.smithi072.stdout:6 upgraded, 23 newly installed, 0 to remove and 250 not upgraded. 2023-10-18T17:56:37.744 INFO:teuthology.orchestra.run.smithi072.stdout:Need to get 10.4 MB of archives. 2023-10-18T17:56:37.744 INFO:teuthology.orchestra.run.smithi072.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-18T17:56:37.744 INFO:teuthology.orchestra.run.smithi072.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-18T17:56:37.838 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:37.878 INFO:teuthology.orchestra.run.smithi072.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-18T17:56:37.880 INFO:teuthology.orchestra.run.smithi072.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-18T17:56:37.882 INFO:teuthology.orchestra.run.smithi072.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-18T17:56:37.887 INFO:teuthology.orchestra.run.smithi072.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-18T17:56:37.887 INFO:teuthology.orchestra.run.smithi072.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-18T17:56:37.905 INFO:teuthology.orchestra.run.smithi072.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-18T17:56:37.906 INFO:teuthology.orchestra.run.smithi072.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-18T17:56:37.907 INFO:teuthology.orchestra.run.smithi072.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-18T17:56:37.909 INFO:teuthology.orchestra.run.smithi072.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-18T17:56:37.909 INFO:teuthology.orchestra.run.smithi072.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-18T17:56:37.911 INFO:teuthology.orchestra.run.smithi072.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-18T17:56:37.912 INFO:teuthology.orchestra.run.smithi072.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-18T17:56:37.914 INFO:teuthology.orchestra.run.smithi072.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-18T17:56:37.914 INFO:teuthology.orchestra.run.smithi072.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-18T17:56:37.928 INFO:teuthology.orchestra.run.smithi072.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-18T17:56:37.929 INFO:teuthology.orchestra.run.smithi072.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-18T17:56:37.939 INFO:teuthology.orchestra.run.smithi072.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-18T17:56:37.947 INFO:teuthology.orchestra.run.smithi072.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-18T17:56:37.947 INFO:teuthology.orchestra.run.smithi072.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-18T17:56:37.964 INFO:teuthology.orchestra.run.smithi072.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-18T17:56:37.964 INFO:teuthology.orchestra.run.smithi072.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-18T17:56:37.973 INFO:teuthology.orchestra.run.smithi072.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-18T17:56:37.981 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mon (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:37.985 INFO:teuthology.orchestra.run.smithi072.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-18T17:56:38.005 INFO:teuthology.orchestra.run.smithi072.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-18T17:56:38.007 INFO:teuthology.orchestra.run.smithi072.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-18T17:56:38.008 INFO:teuthology.orchestra.run.smithi072.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-18T17:56:38.008 INFO:teuthology.orchestra.run.smithi072.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-18T17:56:38.008 INFO:teuthology.orchestra.run.smithi072.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-18T17:56:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-18T17:56:38.206 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-18T17:56:38.435 INFO:teuthology.orchestra.run.smithi072.stdout:Preconfiguring packages ... 2023-10-18T17:56:38.643 INFO:teuthology.orchestra.run.smithi072.stdout:Fetched 10.4 MB in 0s (32.1 MB/s) 2023-10-18T17:56:38.649 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-cephadm (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:38.823 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:38.965 INFO:teuthology.orchestra.run.smithi072.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 115279 files and directories currently installed.) 2023-10-18T17:56:38.969 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-18T17:56:38.999 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ceph-mgr-dashboard (16.2.14-100-gb4ddd52c-1focal) ... 2023-10-18T17:56:39.164 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-18T17:56:39.279 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-18T17:56:39.571 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-18T17:56:40.766 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-18T17:56:42.357 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-18T17:56:42.814 INFO:teuthology.orchestra.run.smithi179.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-18T17:56:42.817 DEBUG:teuthology.orchestra.run.smithi179:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-18T17:56:42.845 INFO:teuthology.orchestra.run.smithi072.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 115279 files and directories currently installed.) 2023-10-18T17:56:42.849 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-18T17:56:42.890 INFO:teuthology.orchestra.run.smithi179.stdout:Reading package lists... 2023-10-18T17:56:42.968 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-18T17:56:43.093 INFO:teuthology.orchestra.run.smithi179.stdout:Building dependency tree... 2023-10-18T17:56:43.094 INFO:teuthology.orchestra.run.smithi179.stdout:Reading state information... 2023-10-18T17:56:43.292 INFO:teuthology.orchestra.run.smithi179.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-18T17:56:43.293 INFO:teuthology.orchestra.run.smithi179.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-18T17:56:43.293 INFO:teuthology.orchestra.run.smithi179.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-18T17:56:43.293 INFO:teuthology.orchestra.run.smithi179.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-18T17:56:43.293 INFO:teuthology.orchestra.run.smithi179.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-18T17:56:43.294 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages were automatically installed and are no longer required: 2023-10-18T17:56:43.295 INFO:teuthology.orchestra.run.smithi179.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-18T17:56:43.295 INFO:teuthology.orchestra.run.smithi179.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-18T17:56:43.295 INFO:teuthology.orchestra.run.smithi179.stdout:The following additional packages will be installed: 2023-10-18T17:56:43.295 INFO:teuthology.orchestra.run.smithi179.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-18T17:56:43.296 INFO:teuthology.orchestra.run.smithi179.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-18T17:56:43.296 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-18T17:56:43.296 INFO:teuthology.orchestra.run.smithi179.stdout: usb-modeswitch-data wpasupplicant 2023-10-18T17:56:43.297 INFO:teuthology.orchestra.run.smithi179.stdout:Suggested packages: 2023-10-18T17:56:43.297 INFO:teuthology.orchestra.run.smithi179.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-18T17:56:43.297 INFO:teuthology.orchestra.run.smithi179.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-18T17:56:43.346 INFO:teuthology.orchestra.run.smithi179.stdout:The following NEW packages will be installed: 2023-10-18T17:56:43.347 INFO:teuthology.orchestra.run.smithi179.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-18T17:56:43.347 INFO:teuthology.orchestra.run.smithi179.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-18T17:56:43.347 INFO:teuthology.orchestra.run.smithi179.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-18T17:56:43.348 INFO:teuthology.orchestra.run.smithi179.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-18T17:56:43.349 INFO:teuthology.orchestra.run.smithi179.stdout:The following packages will be upgraded: 2023-10-18T17:56:43.349 INFO:teuthology.orchestra.run.smithi179.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-18T17:56:43.411 INFO:teuthology.orchestra.run.smithi179.stdout:6 upgraded, 23 newly installed, 0 to remove and 250 not upgraded. 2023-10-18T17:56:43.411 INFO:teuthology.orchestra.run.smithi179.stdout:Need to get 10.4 MB of archives. 2023-10-18T17:56:43.411 INFO:teuthology.orchestra.run.smithi179.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-18T17:56:43.412 INFO:teuthology.orchestra.run.smithi179.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-18T17:56:43.459 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-18T17:56:43.539 INFO:teuthology.orchestra.run.smithi179.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-18T17:56:43.540 INFO:teuthology.orchestra.run.smithi179.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-18T17:56:43.541 INFO:teuthology.orchestra.run.smithi179.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-18T17:56:43.545 INFO:teuthology.orchestra.run.smithi179.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-18T17:56:43.546 INFO:teuthology.orchestra.run.smithi179.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-18T17:56:43.563 INFO:teuthology.orchestra.run.smithi179.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-18T17:56:43.564 INFO:teuthology.orchestra.run.smithi179.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-18T17:56:43.566 INFO:teuthology.orchestra.run.smithi179.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-18T17:56:43.567 INFO:teuthology.orchestra.run.smithi179.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-18T17:56:43.567 INFO:teuthology.orchestra.run.smithi179.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-18T17:56:43.573 INFO:teuthology.orchestra.run.smithi179.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-18T17:56:43.574 INFO:teuthology.orchestra.run.smithi179.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-18T17:56:43.575 INFO:teuthology.orchestra.run.smithi179.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-18T17:56:43.575 INFO:teuthology.orchestra.run.smithi179.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-18T17:56:43.590 INFO:teuthology.orchestra.run.smithi179.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-18T17:56:43.591 INFO:teuthology.orchestra.run.smithi179.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-18T17:56:43.596 INFO:teuthology.orchestra.run.smithi179.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-18T17:56:43.604 INFO:teuthology.orchestra.run.smithi179.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-18T17:56:43.605 INFO:teuthology.orchestra.run.smithi179.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-18T17:56:43.618 INFO:teuthology.orchestra.run.smithi179.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-18T17:56:43.619 INFO:teuthology.orchestra.run.smithi179.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-18T17:56:43.626 INFO:teuthology.orchestra.run.smithi179.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-18T17:56:43.636 INFO:teuthology.orchestra.run.smithi179.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-18T17:56:43.642 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-18T17:56:43.651 INFO:teuthology.orchestra.run.smithi179.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-18T17:56:43.654 INFO:teuthology.orchestra.run.smithi179.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-18T17:56:43.654 INFO:teuthology.orchestra.run.smithi179.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-18T17:56:43.655 INFO:teuthology.orchestra.run.smithi179.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-18T17:56:43.655 INFO:teuthology.orchestra.run.smithi179.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-18T17:56:43.973 INFO:teuthology.orchestra.run.smithi179.stdout:Preconfiguring packages ... 2023-10-18T17:56:44.014 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-18T17:56:44.108 INFO:teuthology.orchestra.run.smithi179.stdout:Fetched 10.4 MB in 0s (35.0 MB/s) 2023-10-18T17:56:44.449 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 112853 files and directories currently installed.) 2023-10-18T17:56:44.452 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-18T17:56:44.558 INFO:teuthology.orchestra.run.smithi072.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 115279 files and directories currently installed.) 2023-10-18T17:56:44.560 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-18T17:56:44.637 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-18T17:56:44.693 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-18T17:56:45.201 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-18T17:56:45.330 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-18T17:56:45.769 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-18T17:56:45.885 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-18T17:56:46.023 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-18T17:56:46.624 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-18T17:56:46.645 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-18T17:56:46.690 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-18T17:56:47.079 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package dump. 2023-10-18T17:56:47.101 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-18T17:56:47.137 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking dump (0.4b46-6) ... 2023-10-18T17:56:47.198 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 112853 files and directories currently installed.) 2023-10-18T17:56:47.202 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-18T17:56:47.309 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-18T17:56:47.479 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package indent. 2023-10-18T17:56:47.501 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-18T17:56:47.545 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-18T17:56:47.721 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-18T17:56:47.932 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-18T17:56:48.057 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-18T17:56:48.077 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-18T17:56:48.122 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-18T17:56:48.266 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-18T17:56:48.530 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-18T17:56:48.551 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-18T17:56:48.597 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-18T17:56:48.704 INFO:teuthology.orchestra.run.smithi179.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 112853 files and directories currently installed.) 2023-10-18T17:56:48.708 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-18T17:56:48.861 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-18T17:56:48.878 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-18T17:56:48.901 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-18T17:56:48.945 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-18T17:56:49.325 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-18T17:56:49.367 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-18T17:56:49.385 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:49.433 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-18T17:56:49.520 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-18T17:56:50.022 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-18T17:56:50.158 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-18T17:56:50.721 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-18T17:56:50.734 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-18T17:56:50.779 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-18T17:56:51.149 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package dump. 2023-10-18T17:56:51.165 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-18T17:56:51.207 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking dump (0.4b46-6) ... 2023-10-18T17:56:51.527 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package indent. 2023-10-18T17:56:51.542 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-18T17:56:51.585 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-18T17:56:51.871 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-18T17:56:51.888 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-18T17:56:51.921 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-18T17:56:52.275 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-18T17:56:52.291 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-18T17:56:52.333 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-18T17:56:52.619 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-18T17:56:52.633 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-18T17:56:52.677 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-18T17:56:53.039 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-18T17:56:53.053 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:53.097 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-18T17:56:53.282 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-18T17:56:53.302 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:53.349 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-18T17:56:53.366 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-18T17:56:53.381 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:53.425 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-18T17:56:53.747 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-18T17:56:53.766 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:53.786 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-18T17:56:53.801 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:53.805 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-18T17:56:53.844 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-18T17:56:54.219 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-18T17:56:54.239 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-18T17:56:54.241 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-18T17:56:54.253 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-18T17:56:54.285 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-18T17:56:54.297 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-18T17:56:54.684 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-18T17:56:54.702 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-18T17:56:54.742 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-18T17:56:54.953 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-18T17:56:54.970 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-18T17:56:55.010 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-18T17:56:55.164 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-18T17:56:55.185 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:55.222 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-18T17:56:55.372 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-18T17:56:55.388 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:55.430 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-18T17:56:55.654 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-18T17:56:55.675 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:55.720 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-18T17:56:55.783 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-18T17:56:55.796 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:55.841 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-18T17:56:55.993 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-18T17:56:56.016 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-18T17:56:56.059 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-18T17:56:56.068 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-18T17:56:56.082 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-18T17:56:56.126 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-18T17:56:56.630 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-18T17:56:56.645 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-18T17:56:56.674 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-18T17:56:56.688 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-18T17:56:56.693 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-18T17:56:56.732 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-18T17:56:57.108 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package modemmanager. 2023-10-18T17:56:57.122 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:57.140 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package modemmanager. 2023-10-18T17:56:57.158 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-18T17:56:57.177 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-18T17:56:57.227 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-18T17:56:57.873 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package wpasupplicant. 2023-10-18T17:56:57.886 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-18T17:56:57.922 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-18T17:56:57.936 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package wpasupplicant. 2023-10-18T17:56:57.954 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-18T17:56:57.994 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-18T17:56:58.487 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package network-manager. 2023-10-18T17:56:58.503 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-18T17:56:58.569 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-18T17:56:59.030 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package network-manager. 2023-10-18T17:56:59.051 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-18T17:56:59.144 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-18T17:56:59.326 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package ppp. 2023-10-18T17:56:59.341 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-18T17:56:59.646 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-18T17:57:00.005 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package ppp. 2023-10-18T17:57:00.028 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-18T17:57:00.246 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package pptp-linux. 2023-10-18T17:57:00.263 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-18T17:57:00.503 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-18T17:57:00.612 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-18T17:57:00.908 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package pptp-linux. 2023-10-18T17:57:00.929 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-18T17:57:00.974 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-18T17:57:01.062 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-18T17:57:01.077 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-18T17:57:01.137 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-18T17:57:01.398 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-18T17:57:01.415 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-18T17:57:01.417 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-18T17:57:01.429 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-18T17:57:01.473 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-18T17:57:01.497 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-18T17:57:01.793 INFO:teuthology.orchestra.run.smithi179.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-18T17:57:01.806 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-18T17:57:01.822 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-18T17:57:01.843 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-18T17:57:01.853 INFO:teuthology.orchestra.run.smithi179.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-18T17:57:01.880 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-18T17:57:02.143 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-18T17:57:02.295 INFO:teuthology.orchestra.run.smithi072.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-18T17:57:02.317 INFO:teuthology.orchestra.run.smithi072.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-18T17:57:02.385 INFO:teuthology.orchestra.run.smithi072.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-18T17:57:02.456 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-18T17:57:02.582 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-18T17:57:02.742 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-18T17:57:02.779 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-18T17:57:03.139 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-18T17:57:03.139 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-18T17:57:03.158 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-18T17:57:03.285 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-18T17:57:03.395 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-18T17:57:03.620 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-18T17:57:03.746 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-18T17:57:03.872 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-18T17:57:03.912 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-18T17:57:03.912 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-18T17:57:04.023 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-18T17:57:04.149 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-18T17:57:04.275 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-18T17:57:04.878 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-18T17:57:05.004 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-18T17:57:05.132 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-18T17:57:05.233 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-18T17:57:05.260 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-18T17:57:05.387 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-18T17:57:05.465 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up dump (0.4b46-6) ... 2023-10-18T17:57:05.514 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-18T17:57:05.560 INFO:teuthology.orchestra.run.smithi179.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-18T17:57:05.624 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-18T17:57:05.750 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-18T17:57:05.986 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-18T17:57:06.086 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-18T17:57:06.229 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-18T17:57:06.339 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-18T17:57:06.498 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-18T17:57:06.624 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-18T17:57:06.792 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-18T17:57:06.810 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-18T17:57:06.918 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-18T17:57:07.226 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up dump (0.4b46-6) ... 2023-10-18T17:57:07.346 INFO:teuthology.orchestra.run.smithi072.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-18T17:57:07.415 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-18T17:57:07.575 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-18T17:57:07.662 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:07.663 INFO:teuthology.orchestra.run.smithi179.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-18T17:57:07.663 INFO:teuthology.orchestra.run.smithi179.stdout:which means they are currently configured by ifupdown: 2023-10-18T17:57:07.663 INFO:teuthology.orchestra.run.smithi179.stdout:- enp3s0f1 2023-10-18T17:57:07.664 INFO:teuthology.orchestra.run.smithi179.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-18T17:57:07.664 INFO:teuthology.orchestra.run.smithi179.stdout:remove their configuration from /etc/network/interfaces. 2023-10-18T17:57:07.664 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:07.702 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-18T17:57:07.821 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-18T17:57:07.837 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-18T17:57:08.024 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-18T17:57:08.145 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-18T17:57:08.152 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-18T17:57:08.295 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-18T17:57:08.422 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-18T17:57:08.626 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-18T17:57:08.644 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-18T17:57:09.129 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-18T17:57:09.273 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-18T17:57:09.415 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-18T17:57:09.420 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-18T17:57:09.541 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-18T17:57:09.709 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-18T17:57:09.835 INFO:teuthology.orchestra.run.smithi179.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-18T17:57:10.116 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-18T17:57:10.117 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-18T17:57:10.621 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-18T17:57:10.641 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:10.641 INFO:teuthology.orchestra.run.smithi072.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-18T17:57:10.641 INFO:teuthology.orchestra.run.smithi072.stdout:which means they are currently configured by ifupdown: 2023-10-18T17:57:10.642 INFO:teuthology.orchestra.run.smithi072.stdout:- enp3s0f1 2023-10-18T17:57:10.642 INFO:teuthology.orchestra.run.smithi072.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-18T17:57:10.642 INFO:teuthology.orchestra.run.smithi072.stdout:remove their configuration from /etc/network/interfaces. 2023-10-18T17:57:10.642 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:10.730 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-18T17:57:10.856 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-18T17:57:10.876 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-18T17:57:11.116 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-18T17:57:11.410 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-18T17:57:11.476 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-18T17:57:12.379 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-18T17:57:13.337 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-18T17:57:13.523 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-18T17:57:13.685 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-18T17:57:13.845 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-18T17:57:13.972 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-18T17:57:14.099 INFO:teuthology.orchestra.run.smithi072.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-18T17:57:14.437 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-18T17:57:14.437 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-18T17:57:15.245 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-18T17:57:15.353 INFO:teuthology.orchestra.run.smithi179.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-18T17:57:15.388 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-18T17:57:15.532 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-18T17:57:15.676 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-18T17:57:15.845 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-18T17:57:16.203 INFO:teuthology.orchestra.run.smithi179.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-18T17:57:16.206 DEBUG:teuthology.parallel:result is None 2023-10-18T17:57:16.333 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-18T17:57:23.270 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-18T17:57:23.379 INFO:teuthology.orchestra.run.smithi072.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-18T17:57:24.682 INFO:teuthology.orchestra.run.smithi072.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-18T17:57:24.689 DEBUG:teuthology.parallel:result is None 2023-10-18T17:57:24.689 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 2023-10-18T17:57:24.840 DEBUG:teuthology.orchestra.run.smithi072:> dpkg-query -W -f '${Version}' ceph 2023-10-18T17:57:24.877 INFO:teuthology.orchestra.run.smithi072.stdout:16.2.14-100-gb4ddd52c-1focal 2023-10-18T17:57:24.878 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-100-gb4ddd52c-1focal 2023-10-18T17:57:24.879 INFO:teuthology.task.install:The correct ceph version 16.2.14-100-gb4ddd52c-1focal is installed. 2023-10-18T17:57:24.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5 2023-10-18T17:57:25.110 DEBUG:teuthology.orchestra.run.smithi179:> dpkg-query -W -f '${Version}' ceph 2023-10-18T17:57:25.133 INFO:teuthology.orchestra.run.smithi179.stdout:16.2.14-100-gb4ddd52c-1focal 2023-10-18T17:57:25.134 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-100-gb4ddd52c-1focal 2023-10-18T17:57:25.134 INFO:teuthology.task.install:The correct ceph version 16.2.14-100-gb4ddd52c-1focal is installed. 2023-10-18T17:57:25.135 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-18T17:57:25.135 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:25.136 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T17:57:25.173 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:57:25.174 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T17:57:25.191 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-18T17:57:25.191 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:25.192 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T17:57:25.248 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T17:57:25.327 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:57:25.328 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T17:57:25.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T17:57:25.354 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-18T17:57:25.354 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:25.355 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T17:57:25.406 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T17:57:25.482 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:57:25.482 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T17:57:25.499 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T17:57:25.508 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-18T17:57:25.509 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:25.509 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T17:57:25.556 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T17:57:25.631 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:57:25.631 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T17:57:25.648 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T17:57:25.707 INFO:teuthology.run_tasks:Running task ceph... 2023-10-18T17:57:25.768 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-18T17:57:25.769 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /var/log/ceph 2023-10-18T17:57:25.777 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /var/log/ceph 2023-10-18T17:57:25.802 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-18T17:57:25.803 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-18T17:57:25.849 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-18T17:57:25.879 INFO:tasks.ceph:Creating extra log directories... 2023-10-18T17:57:25.880 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-18T17:57:25.930 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-18T17:57:25.967 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-18T17:57:25.967 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-18T17:57:25.968 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-18T17:57:25.968 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:25.968 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-18T17:57:26.020 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-18T17:57:26.154 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:57:26.155 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-18T17:57:26.162 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-18T17:57:26.247 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-18T17:57:26.248 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-18T17:57:26.248 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161', 'branch': 'wip-vshankar-testing-pacific-20231016.091648', 'description': 'fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7429161', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-pacific-20231016.091648'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'Replacing daemon mds'], 'sha1': 'b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-pacific-20231016.091648', 'sha1': 'b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 7825, 'sha1': 'b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5', 'sleep_before_teardown': 0, 'subset': '33/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-pacific-20231016.091648', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5', 'targets': {'smithi072.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi179.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5'}}, {'ceph': None}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['suites/iozone.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-16_12:48:28', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-18T17:57:26.249 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-18T17:57:26.257 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-18T17:57:26.270 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/run/ceph 2023-10-18T17:57:26.322 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m0777 -- /var/run/ceph 2023-10-18T17:57:26.351 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:26.351 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T17:57:26.413 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T17:57:26.413 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2023-10-18T17:57:26.479 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T17:57:26.479 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T17:57:26.479 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 498 Links: 1 2023-10-18T17:57:26.480 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T17:57:26.480 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-10-18 17:52:17.827063544 +0000 2023-10-18T17:57:26.480 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-10-18 17:52:17.823063636 +0000 2023-10-18T17:57:26.480 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-10-18 17:52:17.823063636 +0000 2023-10-18T17:57:26.480 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-10-18T17:57:26.481 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T17:57:26.555 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-10-18T17:57:26.555 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-10-18T17:57:26.556 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000580293 s, 882 kB/s 2023-10-18T17:57:26.559 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T17:57:26.628 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2023-10-18T17:57:26.693 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T17:57:26.693 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T17:57:26.693 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 505 Links: 1 2023-10-18T17:57:26.693 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T17:57:26.694 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-10-18 17:52:19.075034763 +0000 2023-10-18T17:57:26.694 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-10-18 17:52:18.595045833 +0000 2023-10-18T17:57:26.694 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-10-18 17:52:18.595045833 +0000 2023-10-18T17:57:26.694 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-10-18T17:57:26.695 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T17:57:26.768 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-10-18T17:57:26.768 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-10-18T17:57:26.769 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000715997 s, 715 kB/s 2023-10-18T17:57:26.770 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T17:57:26.839 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2023-10-18T17:57:26.906 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T17:57:26.906 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T17:57:26.907 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 512 Links: 1 2023-10-18T17:57:26.907 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T17:57:26.907 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-10-18 17:52:19.387027568 +0000 2023-10-18T17:57:26.907 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-10-18 17:52:19.383027661 +0000 2023-10-18T17:57:26.907 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-10-18 17:52:19.383027661 +0000 2023-10-18T17:57:26.908 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-10-18T17:57:26.908 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T17:57:26.981 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-10-18T17:57:26.981 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-10-18T17:57:26.982 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.00063581 s, 805 kB/s 2023-10-18T17:57:26.984 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T17:57:27.054 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2023-10-18T17:57:27.118 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T17:57:27.118 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T17:57:27.119 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 517 Links: 1 2023-10-18T17:57:27.119 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T17:57:27.119 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-10-18 17:52:20.586999895 +0000 2023-10-18T17:57:27.119 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-10-18 17:52:20.127010503 +0000 2023-10-18T17:57:27.119 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-10-18 17:52:20.127010503 +0000 2023-10-18T17:57:27.120 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-10-18T17:57:27.120 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T17:57:27.194 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-10-18T17:57:27.195 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-10-18T17:57:27.195 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000636091 s, 805 kB/s 2023-10-18T17:57:27.196 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T17:57:27.268 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:27.269 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:57:27.269 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T17:57:27.276 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T17:57:27.277 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2023-10-18T17:57:27.326 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T17:57:27.327 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T17:57:27.327 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 358 Links: 1 2023-10-18T17:57:27.327 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T17:57:27.327 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2023-10-18 17:50:35.872664468 +0000 2023-10-18T17:57:27.328 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2023-10-18 17:50:35.592670925 +0000 2023-10-18T17:57:27.328 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2023-10-18 17:50:35.592670925 +0000 2023-10-18T17:57:27.328 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2023-10-18T17:57:27.328 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T17:57:27.343 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2023-10-18T17:57:27.344 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2023-10-18T17:57:27.344 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000391161 s, 1.3 MB/s 2023-10-18T17:57:27.345 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T17:57:27.353 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2023-10-18T17:57:27.402 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T17:57:27.403 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T17:57:27.403 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 365 Links: 1 2023-10-18T17:57:27.403 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T17:57:27.403 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2023-10-18 17:50:36.356653307 +0000 2023-10-18T17:57:27.403 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2023-10-18 17:50:36.096659303 +0000 2023-10-18T17:57:27.404 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2023-10-18 17:50:36.096659303 +0000 2023-10-18T17:57:27.404 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2023-10-18T17:57:27.404 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T17:57:27.415 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2023-10-18T17:57:27.415 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2023-10-18T17:57:27.416 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000376239 s, 1.4 MB/s 2023-10-18T17:57:27.417 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T17:57:27.426 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2023-10-18T17:57:27.478 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T17:57:27.479 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T17:57:27.479 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 373 Links: 1 2023-10-18T17:57:27.479 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T17:57:27.479 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2023-10-18 17:50:36.876641316 +0000 2023-10-18T17:57:27.480 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2023-10-18 17:50:36.592647865 +0000 2023-10-18T17:57:27.480 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2023-10-18 17:50:36.592647865 +0000 2023-10-18T17:57:27.480 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2023-10-18T17:57:27.480 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T17:57:27.493 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2023-10-18T17:57:27.494 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2023-10-18T17:57:27.494 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000395431 s, 1.3 MB/s 2023-10-18T17:57:27.495 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T17:57:27.503 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2023-10-18T17:57:27.555 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T17:57:27.555 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T17:57:27.555 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 6h/6d Inode: 380 Links: 1 2023-10-18T17:57:27.555 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T17:57:27.556 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2023-10-18 17:50:37.392629415 +0000 2023-10-18T17:57:27.556 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2023-10-18 17:50:37.104636058 +0000 2023-10-18T17:57:27.556 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2023-10-18 17:50:37.104636058 +0000 2023-10-18T17:57:27.556 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: - 2023-10-18T17:57:27.557 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T17:57:27.570 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2023-10-18T17:57:27.571 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2023-10-18T17:57:27.571 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000409171 s, 1.3 MB/s 2023-10-18T17:57:27.572 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T17:57:27.624 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:27.624 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi179.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-18T17:57:27.625 INFO:tasks.ceph:Generating config... 2023-10-18T17:57:27.626 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-18T17:57:27.626 INFO:tasks.ceph:[client] debug client = 20 2023-10-18T17:57:27.627 INFO:tasks.ceph:[client] debug ms = 1 2023-10-18T17:57:27.627 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-18T17:57:27.627 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-18T17:57:27.627 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-18T17:57:27.628 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-18T17:57:27.628 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-18T17:57:27.628 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2023-10-18T17:57:27.629 INFO:tasks.ceph:[mds] mds bal merge size = 5 2023-10-18T17:57:27.629 INFO:tasks.ceph:[mds] mds bal split bits = 3 2023-10-18T17:57:27.629 INFO:tasks.ceph:[mds] mds bal split size = 100 2023-10-18T17:57:27.630 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-18T17:57:27.630 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-18T17:57:27.630 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = False 2023-10-18T17:57:27.630 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-18T17:57:27.631 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-18T17:57:27.631 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2023-10-18T17:57:27.631 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-18T17:57:27.631 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-18T17:57:27.632 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-18T17:57:27.632 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-18T17:57:27.632 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-18T17:57:27.633 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-18T17:57:27.633 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-18T17:57:27.633 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-18T17:57:27.633 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-18T17:57:27.634 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-18T17:57:27.634 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-18T17:57:27.634 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-18T17:57:27.634 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-18T17:57:27.635 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-18T17:57:27.635 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-18T17:57:27.635 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-18T17:57:27.635 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-18T17:57:27.636 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-18T17:57:27.636 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-18T17:57:27.636 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-18T17:57:27.636 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-18T17:57:27.637 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-18T17:57:27.637 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-18T17:57:27.637 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-10-18T17:57:27.638 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-18T17:57:27.638 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-18T17:57:27.638 INFO:tasks.ceph:Setting up mon.a... 2023-10-18T17:57:27.639 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-18T17:57:27.719 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.keyring 2023-10-18T17:57:27.725 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-18T17:57:27.788 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T17:57:27.822 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.72'), ('mon.b', '172.21.15.179'), ('mon.c', '[v2:172.21.15.179:3301,v1:172.21.15.179:6790]')] 2023-10-18T17:57:27.823 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.72,172.21.15.179,[v2:172.21.15.179:3301,v1:172.21.15.179:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-18T17:57:27.824 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:27.824 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T17:57:27.882 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.72 --add b 172.21.15.179 --addv c '[v2:172.21.15.179:3301,v1:172.21.15.179:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:27.980 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:27.981 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: generated fsid 42f1988f-3a5a-4222-b2d6-b1152a417ca8 2023-10-18T17:57:27.981 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 0 2023-10-18T17:57:27.981 INFO:teuthology.orchestra.run.smithi072.stdout:fsid 42f1988f-3a5a-4222-b2d6-b1152a417ca8 2023-10-18T17:57:27.982 INFO:teuthology.orchestra.run.smithi072.stdout:last_changed 2023-10-18T17:57:27.980985+0000 2023-10-18T17:57:27.982 INFO:teuthology.orchestra.run.smithi072.stdout:created 2023-10-18T17:57:27.980985+0000 2023-10-18T17:57:27.982 INFO:teuthology.orchestra.run.smithi072.stdout:min_mon_release 0 (unknown) 2023-10-18T17:57:27.982 INFO:teuthology.orchestra.run.smithi072.stdout:election_strategy: 1 2023-10-18T17:57:27.982 INFO:teuthology.orchestra.run.smithi072.stdout:0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2023-10-18T17:57:27.983 INFO:teuthology.orchestra.run.smithi072.stdout:1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.b 2023-10-18T17:57:27.983 INFO:teuthology.orchestra.run.smithi072.stdout:2: [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] mon.c 2023-10-18T17:57:27.983 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-18T17:57:27.984 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T17:57:28.004 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 42f1988f-3a5a-4222-b2d6-b1152a417ca8... 2023-10-18T17:57:28.005 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-18T17:57:28.058 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-18T17:57:28.094 INFO:teuthology.orchestra.run.smithi179.stdout:[global] 2023-10-18T17:57:28.095 INFO:teuthology.orchestra.run.smithi179.stdout: chdir = "" 2023-10-18T17:57:28.095 INFO:teuthology.orchestra.run.smithi179.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-18T17:57:28.095 INFO:teuthology.orchestra.run.smithi179.stdout: auth supported = cephx 2023-10-18T17:57:28.095 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.096 INFO:teuthology.orchestra.run.smithi179.stdout: filestore xattr use omap = true 2023-10-18T17:57:28.096 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.096 INFO:teuthology.orchestra.run.smithi179.stdout: mon clock drift allowed = 1.000 2023-10-18T17:57:28.096 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.096 INFO:teuthology.orchestra.run.smithi179.stdout: osd crush chooseleaf type = 0 2023-10-18T17:57:28.097 INFO:teuthology.orchestra.run.smithi179.stdout: auth debug = true 2023-10-18T17:57:28.097 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.097 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on old message = true 2023-10-18T17:57:28.097 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on bug = true 2023-10-18T17:57:28.097 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-18T17:57:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: mon pg warn max object skew = 0 2023-10-18T17:57:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: # disable pg_autoscaler by default for new pools 2023-10-18T17:57:28.098 INFO:teuthology.orchestra.run.smithi179.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-18T17:57:28.099 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.099 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default size = 2 2023-10-18T17:57:28.099 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.099 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow primary affinity = true 2023-10-18T17:57:28.099 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd allow pg remap = true 2023-10-18T17:57:28.100 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on legacy crush tunables = false 2023-10-18T17:57:28.100 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on crush straw calc version zero = false 2023-10-18T17:57:28.100 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on no sortbitwise = false 2023-10-18T17:57:28.100 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on osd down out interval zero = false 2023-10-18T17:57:28.100 INFO:teuthology.orchestra.run.smithi179.stdout: mon warn on too few osds = false 2023-10-18T17:57:28.100 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-18T17:57:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-18T17:57:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: mon_allow_pool_size_one = true 2023-10-18T17:57:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-10-18T17:57:28.101 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.102 INFO:teuthology.orchestra.run.smithi179.stdout: osd default data pool replay window = 5 2023-10-18T17:57:28.102 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.102 INFO:teuthology.orchestra.run.smithi179.stdout: mon allow pool delete = true 2023-10-18T17:57:28.102 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.102 INFO:teuthology.orchestra.run.smithi179.stdout: mon cluster log file level = debug 2023-10-18T17:57:28.103 INFO:teuthology.orchestra.run.smithi179.stdout: debug asserts on shutdown = true 2023-10-18T17:57:28.103 INFO:teuthology.orchestra.run.smithi179.stdout: mon health detail to clog = false 2023-10-18T17:57:28.103 INFO:teuthology.orchestra.run.smithi179.stdout: mon host = "172.21.15.72,172.21.15.179,[v2:172.21.15.179:3301,v1:172.21.15.179:6790]" 2023-10-18T17:57:28.103 INFO:teuthology.orchestra.run.smithi179.stdout: ms die on skipped message = False 2023-10-18T17:57:28.103 INFO:teuthology.orchestra.run.smithi179.stdout: fsid = 42f1988f-3a5a-4222-b2d6-b1152a417ca8 2023-10-18T17:57:28.104 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.104 INFO:teuthology.orchestra.run.smithi179.stdout:[osd] 2023-10-18T17:57:28.104 INFO:teuthology.orchestra.run.smithi179.stdout: osd journal size = 100 2023-10-18T17:57:28.104 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.104 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub load threshold = 5.0 2023-10-18T17:57:28.105 INFO:teuthology.orchestra.run.smithi179.stdout: osd scrub max interval = 600 2023-10-18T17:57:28.105 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.105 INFO:teuthology.orchestra.run.smithi179.stdout: osd recover clone overlap = true 2023-10-18T17:57:28.105 INFO:teuthology.orchestra.run.smithi179.stdout: osd recovery max chunk = 1048576 2023-10-18T17:57:28.105 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.105 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug shutdown = true 2023-10-18T17:57:28.106 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug op order = true 2023-10-18T17:57:28.106 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug verify stray on activate = true 2023-10-18T17:57:28.106 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.106 INFO:teuthology.orchestra.run.smithi179.stdout: osd open classes on start = true 2023-10-18T17:57:28.106 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug pg log writeout = true 2023-10-18T17:57:28.107 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.107 INFO:teuthology.orchestra.run.smithi179.stdout: osd deep scrub update digest min age = 30 2023-10-18T17:57:28.107 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.107 INFO:teuthology.orchestra.run.smithi179.stdout: osd map max advance = 10 2023-10-18T17:57:28.107 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.108 INFO:teuthology.orchestra.run.smithi179.stdout: journal zero on create = true 2023-10-18T17:57:28.108 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.108 INFO:teuthology.orchestra.run.smithi179.stdout: filestore ondisk finisher threads = 3 2023-10-18T17:57:28.108 INFO:teuthology.orchestra.run.smithi179.stdout: filestore apply finisher threads = 3 2023-10-18T17:57:28.108 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.108 INFO:teuthology.orchestra.run.smithi179.stdout: bdev debug aio = true 2023-10-18T17:57:28.109 INFO:teuthology.orchestra.run.smithi179.stdout: osd debug misdirected ops = true 2023-10-18T17:57:28.109 INFO:teuthology.orchestra.run.smithi179.stdout: bdev async discard = True 2023-10-18T17:57:28.109 INFO:teuthology.orchestra.run.smithi179.stdout: bdev enable discard = True 2023-10-18T17:57:28.109 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore allocator = bitmap 2023-10-18T17:57:28.110 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore block size = 96636764160 2023-10-18T17:57:28.110 INFO:teuthology.orchestra.run.smithi179.stdout: bluestore fsck on mount = True 2023-10-18T17:57:28.110 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluefs = 1/20 2023-10-18T17:57:28.110 INFO:teuthology.orchestra.run.smithi179.stdout: debug bluestore = 1/20 2023-10-18T17:57:28.110 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2023-10-18T17:57:28.110 INFO:teuthology.orchestra.run.smithi179.stdout: debug osd = 20 2023-10-18T17:57:28.111 INFO:teuthology.orchestra.run.smithi179.stdout: debug rocksdb = 4/10 2023-10-18T17:57:28.111 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd backfillfull_ratio = 0.85 2023-10-18T17:57:28.111 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd full ratio = 0.9 2023-10-18T17:57:28.111 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd nearfull ratio = 0.8 2023-10-18T17:57:28.111 INFO:teuthology.orchestra.run.smithi179.stdout: osd failsafe full ratio = 0.95 2023-10-18T17:57:28.112 INFO:teuthology.orchestra.run.smithi179.stdout: osd max markdown count = 1000 2023-10-18T17:57:28.112 INFO:teuthology.orchestra.run.smithi179.stdout: osd objectstore = bluestore 2023-10-18T17:57:28.112 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2023-10-18T17:57:28.112 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.112 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr] 2023-10-18T17:57:28.113 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2023-10-18T17:57:28.113 INFO:teuthology.orchestra.run.smithi179.stdout: debug mgr = 20 2023-10-18T17:57:28.113 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2023-10-18T17:57:28.113 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2023-10-18T17:57:28.113 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2023-10-18T17:57:28.114 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2023-10-18T17:57:28.114 INFO:teuthology.orchestra.run.smithi179.stdout: mgr/telemetry/nag = false 2023-10-18T17:57:28.114 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.114 INFO:teuthology.orchestra.run.smithi179.stdout:[mon] 2023-10-18T17:57:28.114 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2023-10-18T17:57:28.115 INFO:teuthology.orchestra.run.smithi179.stdout: debug mon = 20 2023-10-18T17:57:28.115 INFO:teuthology.orchestra.run.smithi179.stdout: debug paxos = 20 2023-10-18T17:57:28.115 INFO:teuthology.orchestra.run.smithi179.stdout: debug auth = 20 2023-10-18T17:57:28.115 INFO:teuthology.orchestra.run.smithi179.stdout: mon data avail warn = 5 2023-10-18T17:57:28.115 INFO:teuthology.orchestra.run.smithi179.stdout: mon mgr mkfs grace = 240 2023-10-18T17:57:28.116 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min pgs per osd = 4 2023-10-18T17:57:28.116 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd reporter subtree level = osd 2023-10-18T17:57:28.116 INFO:teuthology.orchestra.run.smithi179.stdout: mon osd prime pg temp = true 2023-10-18T17:57:28.116 INFO:teuthology.orchestra.run.smithi179.stdout: mon reweight min bytes per osd = 10 2023-10-18T17:57:28.116 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.117 INFO:teuthology.orchestra.run.smithi179.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-18T17:57:28.117 INFO:teuthology.orchestra.run.smithi179.stdout: auth mon ticket ttl = 660 # 11m 2023-10-18T17:57:28.117 INFO:teuthology.orchestra.run.smithi179.stdout: auth service ticket ttl = 240 # 4m 2023-10-18T17:57:28.117 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.117 INFO:teuthology.orchestra.run.smithi179.stdout: # don't complain about insecure global_id in the test suite 2023-10-18T17:57:28.118 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-18T17:57:28.118 INFO:teuthology.orchestra.run.smithi179.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-18T17:57:28.118 INFO:teuthology.orchestra.run.smithi179.stdout: mon op complaint time = 120 2023-10-18T17:57:28.118 INFO:teuthology.orchestra.run.smithi179.stdout: 2023-10-18T17:57:28.118 INFO:teuthology.orchestra.run.smithi179.stdout:[client] 2023-10-18T17:57:28.119 INFO:teuthology.orchestra.run.smithi179.stdout: rgw cache enabled = true 2023-10-18T17:57:28.119 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable ops log = true 2023-10-18T17:57:28.119 INFO:teuthology.orchestra.run.smithi179.stdout: rgw enable usage log = true 2023-10-18T17:57:28.119 INFO:teuthology.orchestra.run.smithi179.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-18T17:57:28.119 INFO:teuthology.orchestra.run.smithi179.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-18T17:57:28.119 INFO:teuthology.orchestra.run.smithi179.stdout: client mount timeout = 600 2023-10-18T17:57:28.120 INFO:teuthology.orchestra.run.smithi179.stdout: debug client = 20 2023-10-18T17:57:28.120 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2023-10-18T17:57:28.120 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2023-10-18T17:57:28.120 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2023-10-18T17:57:28.120 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.a] 2023-10-18T17:57:28.121 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.b] 2023-10-18T17:57:28.121 INFO:teuthology.orchestra.run.smithi179.stdout:[mon.c] 2023-10-18T17:57:28.121 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2023-10-18T17:57:28.121 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2023-10-18T17:57:28.121 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.e] 2023-10-18T17:57:28.122 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2023-10-18T17:57:28.122 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2023-10-18T17:57:28.122 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.f] 2023-10-18T17:57:28.122 INFO:teuthology.orchestra.run.smithi179.stdout:[mds] 2023-10-18T17:57:28.122 INFO:teuthology.orchestra.run.smithi179.stdout: debug mds = 20 2023-10-18T17:57:28.123 INFO:teuthology.orchestra.run.smithi179.stdout: debug ms = 1 2023-10-18T17:57:28.123 INFO:teuthology.orchestra.run.smithi179.stdout: mds bal fragment size max = 10000 2023-10-18T17:57:28.123 INFO:teuthology.orchestra.run.smithi179.stdout: mds bal merge size = 5 2023-10-18T17:57:28.123 INFO:teuthology.orchestra.run.smithi179.stdout: mds bal split bits = 3 2023-10-18T17:57:28.123 INFO:teuthology.orchestra.run.smithi179.stdout: mds bal split size = 100 2023-10-18T17:57:28.124 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug frag = True 2023-10-18T17:57:28.124 INFO:teuthology.orchestra.run.smithi179.stdout: mds debug scatterstat = True 2023-10-18T17:57:28.124 INFO:teuthology.orchestra.run.smithi179.stdout: mds oft prefetch dirfrags = False 2023-10-18T17:57:28.124 INFO:teuthology.orchestra.run.smithi179.stdout: mds op complaint time = 180 2023-10-18T17:57:28.124 INFO:teuthology.orchestra.run.smithi179.stdout: mds verify scatter = True 2023-10-18T17:57:28.125 INFO:teuthology.orchestra.run.smithi179.stdout: mds_sleep_rank_change = 5000000.0 2023-10-18T17:57:28.125 INFO:teuthology.orchestra.run.smithi179.stdout: osd op complaint time = 180 2023-10-18T17:57:28.125 INFO:teuthology.orchestra.run.smithi179.stdout: rados mon op timeout = 900 2023-10-18T17:57:28.125 INFO:teuthology.orchestra.run.smithi179.stdout: rados osd op timeout = 900 2023-10-18T17:57:28.132 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2023-10-18T17:57:28.132 INFO:teuthology.orchestra.run.smithi072.stdout: chdir = "" 2023-10-18T17:57:28.133 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-18T17:57:28.133 INFO:teuthology.orchestra.run.smithi072.stdout: auth supported = cephx 2023-10-18T17:57:28.133 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.133 INFO:teuthology.orchestra.run.smithi072.stdout: filestore xattr use omap = true 2023-10-18T17:57:28.133 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.134 INFO:teuthology.orchestra.run.smithi072.stdout: mon clock drift allowed = 1.000 2023-10-18T17:57:28.134 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.134 INFO:teuthology.orchestra.run.smithi072.stdout: osd crush chooseleaf type = 0 2023-10-18T17:57:28.134 INFO:teuthology.orchestra.run.smithi072.stdout: auth debug = true 2023-10-18T17:57:28.134 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.135 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on old message = true 2023-10-18T17:57:28.135 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on bug = true 2023-10-18T17:57:28.135 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.135 INFO:teuthology.orchestra.run.smithi072.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-18T17:57:28.136 INFO:teuthology.orchestra.run.smithi072.stdout: mon pg warn max object skew = 0 2023-10-18T17:57:28.136 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.136 INFO:teuthology.orchestra.run.smithi072.stdout: # disable pg_autoscaler by default for new pools 2023-10-18T17:57:28.136 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-18T17:57:28.136 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.137 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default size = 2 2023-10-18T17:57:28.137 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.137 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow primary affinity = true 2023-10-18T17:57:28.137 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow pg remap = true 2023-10-18T17:57:28.137 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on legacy crush tunables = false 2023-10-18T17:57:28.138 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on crush straw calc version zero = false 2023-10-18T17:57:28.138 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on no sortbitwise = false 2023-10-18T17:57:28.138 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on osd down out interval zero = false 2023-10-18T17:57:28.138 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on too few osds = false 2023-10-18T17:57:28.138 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-18T17:57:28.139 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-18T17:57:28.139 INFO:teuthology.orchestra.run.smithi072.stdout: mon_allow_pool_size_one = true 2023-10-18T17:57:28.139 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.139 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-10-18T17:57:28.139 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.139 INFO:teuthology.orchestra.run.smithi072.stdout: osd default data pool replay window = 5 2023-10-18T17:57:28.140 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.140 INFO:teuthology.orchestra.run.smithi072.stdout: mon allow pool delete = true 2023-10-18T17:57:28.140 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.140 INFO:teuthology.orchestra.run.smithi072.stdout: mon cluster log file level = debug 2023-10-18T17:57:28.140 INFO:teuthology.orchestra.run.smithi072.stdout: debug asserts on shutdown = true 2023-10-18T17:57:28.141 INFO:teuthology.orchestra.run.smithi072.stdout: mon health detail to clog = false 2023-10-18T17:57:28.141 INFO:teuthology.orchestra.run.smithi072.stdout: mon host = "172.21.15.72,172.21.15.179,[v2:172.21.15.179:3301,v1:172.21.15.179:6790]" 2023-10-18T17:57:28.141 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on skipped message = False 2023-10-18T17:57:28.141 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = 42f1988f-3a5a-4222-b2d6-b1152a417ca8 2023-10-18T17:57:28.141 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.141 INFO:teuthology.orchestra.run.smithi072.stdout:[osd] 2023-10-18T17:57:28.142 INFO:teuthology.orchestra.run.smithi072.stdout: osd journal size = 100 2023-10-18T17:57:28.142 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.142 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub load threshold = 5.0 2023-10-18T17:57:28.142 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub max interval = 600 2023-10-18T17:57:28.142 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.143 INFO:teuthology.orchestra.run.smithi072.stdout: osd recover clone overlap = true 2023-10-18T17:57:28.143 INFO:teuthology.orchestra.run.smithi072.stdout: osd recovery max chunk = 1048576 2023-10-18T17:57:28.143 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.143 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug shutdown = true 2023-10-18T17:57:28.143 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug op order = true 2023-10-18T17:57:28.144 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify stray on activate = true 2023-10-18T17:57:28.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.144 INFO:teuthology.orchestra.run.smithi072.stdout: osd open classes on start = true 2023-10-18T17:57:28.144 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug pg log writeout = true 2023-10-18T17:57:28.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.145 INFO:teuthology.orchestra.run.smithi072.stdout: osd deep scrub update digest min age = 30 2023-10-18T17:57:28.145 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.145 INFO:teuthology.orchestra.run.smithi072.stdout: osd map max advance = 10 2023-10-18T17:57:28.145 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.145 INFO:teuthology.orchestra.run.smithi072.stdout: journal zero on create = true 2023-10-18T17:57:28.145 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.146 INFO:teuthology.orchestra.run.smithi072.stdout: filestore ondisk finisher threads = 3 2023-10-18T17:57:28.146 INFO:teuthology.orchestra.run.smithi072.stdout: filestore apply finisher threads = 3 2023-10-18T17:57:28.146 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.146 INFO:teuthology.orchestra.run.smithi072.stdout: bdev debug aio = true 2023-10-18T17:57:28.146 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug misdirected ops = true 2023-10-18T17:57:28.147 INFO:teuthology.orchestra.run.smithi072.stdout: bdev async discard = True 2023-10-18T17:57:28.147 INFO:teuthology.orchestra.run.smithi072.stdout: bdev enable discard = True 2023-10-18T17:57:28.147 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore allocator = bitmap 2023-10-18T17:57:28.147 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore block size = 96636764160 2023-10-18T17:57:28.147 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore fsck on mount = True 2023-10-18T17:57:28.148 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluefs = 1/20 2023-10-18T17:57:28.148 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluestore = 1/20 2023-10-18T17:57:28.148 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-18T17:57:28.148 INFO:teuthology.orchestra.run.smithi072.stdout: debug osd = 20 2023-10-18T17:57:28.148 INFO:teuthology.orchestra.run.smithi072.stdout: debug rocksdb = 4/10 2023-10-18T17:57:28.148 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd backfillfull_ratio = 0.85 2023-10-18T17:57:28.149 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd full ratio = 0.9 2023-10-18T17:57:28.149 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd nearfull ratio = 0.8 2023-10-18T17:57:28.149 INFO:teuthology.orchestra.run.smithi072.stdout: osd failsafe full ratio = 0.95 2023-10-18T17:57:28.149 INFO:teuthology.orchestra.run.smithi072.stdout: osd max markdown count = 1000 2023-10-18T17:57:28.150 INFO:teuthology.orchestra.run.smithi072.stdout: osd objectstore = bluestore 2023-10-18T17:57:28.150 INFO:teuthology.orchestra.run.smithi072.stdout: osd op complaint time = 180 2023-10-18T17:57:28.150 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.150 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr] 2023-10-18T17:57:28.150 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-18T17:57:28.151 INFO:teuthology.orchestra.run.smithi072.stdout: debug mgr = 20 2023-10-18T17:57:28.151 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-10-18T17:57:28.151 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-10-18T17:57:28.151 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-10-18T17:57:28.151 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-10-18T17:57:28.152 INFO:teuthology.orchestra.run.smithi072.stdout: mgr/telemetry/nag = false 2023-10-18T17:57:28.152 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.152 INFO:teuthology.orchestra.run.smithi072.stdout:[mon] 2023-10-18T17:57:28.152 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-18T17:57:28.152 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-10-18T17:57:28.153 INFO:teuthology.orchestra.run.smithi072.stdout: debug paxos = 20 2023-10-18T17:57:28.153 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-10-18T17:57:28.153 INFO:teuthology.orchestra.run.smithi072.stdout: mon data avail warn = 5 2023-10-18T17:57:28.153 INFO:teuthology.orchestra.run.smithi072.stdout: mon mgr mkfs grace = 240 2023-10-18T17:57:28.153 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-10-18T17:57:28.153 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd reporter subtree level = osd 2023-10-18T17:57:28.154 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd prime pg temp = true 2023-10-18T17:57:28.154 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-10-18T17:57:28.154 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.154 INFO:teuthology.orchestra.run.smithi072.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-18T17:57:28.154 INFO:teuthology.orchestra.run.smithi072.stdout: auth mon ticket ttl = 660 # 11m 2023-10-18T17:57:28.155 INFO:teuthology.orchestra.run.smithi072.stdout: auth service ticket ttl = 240 # 4m 2023-10-18T17:57:28.155 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.155 INFO:teuthology.orchestra.run.smithi072.stdout: # don't complain about insecure global_id in the test suite 2023-10-18T17:57:28.155 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-18T17:57:28.155 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-18T17:57:28.156 INFO:teuthology.orchestra.run.smithi072.stdout: mon op complaint time = 120 2023-10-18T17:57:28.156 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:57:28.156 INFO:teuthology.orchestra.run.smithi072.stdout:[client] 2023-10-18T17:57:28.156 INFO:teuthology.orchestra.run.smithi072.stdout: rgw cache enabled = true 2023-10-18T17:57:28.156 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable ops log = true 2023-10-18T17:57:28.156 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable usage log = true 2023-10-18T17:57:28.157 INFO:teuthology.orchestra.run.smithi072.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-18T17:57:28.157 INFO:teuthology.orchestra.run.smithi072.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-18T17:57:28.157 INFO:teuthology.orchestra.run.smithi072.stdout: client mount timeout = 600 2023-10-18T17:57:28.157 INFO:teuthology.orchestra.run.smithi072.stdout: debug client = 20 2023-10-18T17:57:28.157 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-18T17:57:28.158 INFO:teuthology.orchestra.run.smithi072.stdout: rados mon op timeout = 900 2023-10-18T17:57:28.158 INFO:teuthology.orchestra.run.smithi072.stdout: rados osd op timeout = 900 2023-10-18T17:57:28.158 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.a] 2023-10-18T17:57:28.158 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.b] 2023-10-18T17:57:28.158 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.c] 2023-10-18T17:57:28.159 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a] 2023-10-18T17:57:28.159 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c] 2023-10-18T17:57:28.159 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.e] 2023-10-18T17:57:28.159 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b] 2023-10-18T17:57:28.159 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d] 2023-10-18T17:57:28.160 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.f] 2023-10-18T17:57:28.160 INFO:teuthology.orchestra.run.smithi072.stdout:[mds] 2023-10-18T17:57:28.160 INFO:teuthology.orchestra.run.smithi072.stdout: debug mds = 20 2023-10-18T17:57:28.160 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-10-18T17:57:28.160 INFO:teuthology.orchestra.run.smithi072.stdout: mds bal fragment size max = 10000 2023-10-18T17:57:28.160 INFO:teuthology.orchestra.run.smithi072.stdout: mds bal merge size = 5 2023-10-18T17:57:28.161 INFO:teuthology.orchestra.run.smithi072.stdout: mds bal split bits = 3 2023-10-18T17:57:28.161 INFO:teuthology.orchestra.run.smithi072.stdout: mds bal split size = 100 2023-10-18T17:57:28.161 INFO:teuthology.orchestra.run.smithi072.stdout: mds debug frag = True 2023-10-18T17:57:28.161 INFO:teuthology.orchestra.run.smithi072.stdout: mds debug scatterstat = True 2023-10-18T17:57:28.161 INFO:teuthology.orchestra.run.smithi072.stdout: mds oft prefetch dirfrags = False 2023-10-18T17:57:28.162 INFO:teuthology.orchestra.run.smithi072.stdout: mds op complaint time = 180 2023-10-18T17:57:28.162 INFO:teuthology.orchestra.run.smithi072.stdout: mds verify scatter = True 2023-10-18T17:57:28.162 INFO:teuthology.orchestra.run.smithi072.stdout: mds_sleep_rank_change = 5000000.0 2023-10-18T17:57:28.162 INFO:teuthology.orchestra.run.smithi072.stdout: osd op complaint time = 180 2023-10-18T17:57:28.162 INFO:teuthology.orchestra.run.smithi072.stdout: rados mon op timeout = 900 2023-10-18T17:57:28.163 INFO:teuthology.orchestra.run.smithi072.stdout: rados osd op timeout = 900 2023-10-18T17:57:28.164 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-18T17:57:28.164 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-18T17:57:28.281 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-18T17:57:28.282 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:28.282 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-18T17:57:28.301 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:28.301 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-18T17:57:28.360 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T17:57:28.360 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:28.361 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-18T17:57:28.361 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T17:57:28.463 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:57:28.463 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:28.523 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T17:57:28.524 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:57:28.524 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-18T17:57:28.524 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T17:57:28.551 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:57:28.552 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:28.557 INFO:tasks.ceph:Setting up mon nodes... 2023-10-18T17:57:28.558 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-18T17:57:28.558 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-18T17:57:28.660 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-18T17:57:28.667 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T17:57:28.736 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T17:57:28.739 INFO:tasks.ceph:Setting up mds nodes... 2023-10-18T17:57:28.739 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-18T17:57:28.831 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-18T17:57:28.836 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-18T17:57:28.870 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-18T17:57:29.000 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-18T17:57:29.006 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-18T17:57:29.040 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2023-10-18T17:57:29.164 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2023-10-18T17:57:29.167 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2023-10-18T17:57:29.200 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-18T17:57:29.244 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-18T17:57:29.247 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-18T17:57:29.260 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-18T17:57:29.346 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-18T17:57:29.348 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-18T17:57:29.361 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2023-10-18T17:57:29.401 INFO:teuthology.orchestra.run.smithi179.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2023-10-18T17:57:29.403 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2023-10-18T17:57:29.417 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-18T17:57:29.417 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-18T17:57:29.479 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-18T17:57:29.550 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-18T17:57:29.551 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi179.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-18T17:57:29.551 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-18T17:57:29.579 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:29.580 INFO:tasks.ceph:role: osd.0 2023-10-18T17:57:29.580 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T17:57:29.580 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-18T17:57:30.184 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T17:57:30.184 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T17:57:30.184 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T17:57:30.184 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 2023-10-18T17:57:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T17:57:30.185 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-10-18T17:57:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T17:57:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T17:57:30.185 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T17:57:30.185 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T17:57:30.189 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-10-18T17:57:30.190 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-18T17:57:30.333 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-18T17:57:30.356 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T17:57:30.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:57:30.357 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-18T17:57:30.433 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:30.433 INFO:tasks.ceph:role: osd.1 2023-10-18T17:57:30.434 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T17:57:30.434 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-18T17:57:30.946 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T17:57:30.947 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T17:57:30.947 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T17:57:30.947 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 2023-10-18T17:57:30.947 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T17:57:30.947 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-10-18T17:57:30.948 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T17:57:30.948 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T17:57:30.948 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T17:57:30.948 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T17:57:30.951 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-10-18T17:57:30.951 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-18T17:57:31.022 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-18T17:57:31.045 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T17:57:31.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:57:31.047 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-18T17:57:31.128 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:31.128 INFO:tasks.ceph:role: osd.2 2023-10-18T17:57:31.129 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T17:57:31.129 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-18T17:57:31.702 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T17:57:31.702 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T17:57:31.702 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T17:57:31.703 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 2023-10-18T17:57:31.703 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T17:57:31.703 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-10-18T17:57:31.703 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T17:57:31.703 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T17:57:31.704 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T17:57:31.704 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T17:57:31.707 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-10-18T17:57:31.707 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-18T17:57:31.844 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-18T17:57:31.867 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T17:57:31.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:57:31.869 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-18T17:57:31.951 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:31.951 INFO:tasks.ceph:role: osd.3 2023-10-18T17:57:31.951 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T17:57:31.952 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-18T17:57:32.520 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T17:57:32.521 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T17:57:32.521 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T17:57:32.521 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 2023-10-18T17:57:32.521 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T17:57:32.522 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-10-18T17:57:32.522 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T17:57:32.522 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T17:57:32.522 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T17:57:32.522 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T17:57:32.524 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-10-18T17:57:32.524 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-18T17:57:32.735 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-18T17:57:32.763 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T17:57:32.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:57:32.765 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:32.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:32.920+0000 7f28b14e9080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-18T17:57:32.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:32.924+0000 7f28b14e9080 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-18T17:57:32.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:32.924+0000 7f28b14e9080 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T17:57:32.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:32.924+0000 7f28b14e9080 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T17:57:32.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:32.924+0000 7f28b14e9080 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-18T17:57:32.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:32.928+0000 7f28b14e9080 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-18T17:57:36.210 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-18T17:57:36.245 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:36.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:36.360+0000 7fa6f24e2080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-18T17:57:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:36.364+0000 7fa6f24e2080 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-18T17:57:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:36.364+0000 7fa6f24e2080 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T17:57:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:36.364+0000 7fa6f24e2080 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T17:57:36.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:36.364+0000 7fa6f24e2080 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-18T17:57:36.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:36.368+0000 7fa6f24e2080 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-18T17:57:39.918 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-18T17:57:39.956 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:40.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:40.072+0000 7fbaaceda080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-18T17:57:40.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:40.072+0000 7fbaaceda080 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-18T17:57:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:40.072+0000 7fbaaceda080 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T17:57:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:40.072+0000 7fbaaceda080 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T17:57:40.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:40.072+0000 7fbaaceda080 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-18T17:57:40.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:40.076+0000 7fbaaceda080 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-18T17:57:43.623 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-18T17:57:43.656 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:43.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:43.767+0000 7f81b42f4080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-18T17:57:43.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:43.771+0000 7f81b42f4080 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-18T17:57:43.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:43.771+0000 7f81b42f4080 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-18T17:57:43.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:43.771+0000 7f81b42f4080 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-18T17:57:43.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:43.771+0000 7f81b42f4080 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-18T17:57:43.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:57:43.771+0000 7f81b42f4080 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-18T17:57:47.051 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-18T17:57:47.088 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-18T17:57:47.106 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:47.106 INFO:tasks.ceph:role: osd.4 2023-10-18T17:57:47.107 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T17:57:47.107 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-18T17:57:47.632 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T17:57:47.633 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T17:57:47.633 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T17:57:47.633 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 2023-10-18T17:57:47.633 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T17:57:47.634 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2023-10-18T17:57:47.634 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T17:57:47.634 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T17:57:47.634 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T17:57:47.634 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T17:57:47.635 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2023-10-18T17:57:47.636 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-18T17:57:47.723 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-18T17:57:47.732 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T17:57:47.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:57:47.734 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-18T17:57:47.748 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:47.749 INFO:tasks.ceph:role: osd.5 2023-10-18T17:57:47.749 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T17:57:47.749 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-18T17:57:48.282 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T17:57:48.283 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T17:57:48.283 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T17:57:48.283 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 2023-10-18T17:57:48.283 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T17:57:48.284 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2023-10-18T17:57:48.284 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T17:57:48.284 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T17:57:48.285 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T17:57:48.285 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T17:57:48.286 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2023-10-18T17:57:48.286 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-18T17:57:48.337 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-18T17:57:48.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:57:48.350 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T17:57:48.351 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-18T17:57:48.364 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:48.365 INFO:tasks.ceph:role: osd.6 2023-10-18T17:57:48.365 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T17:57:48.365 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-18T17:57:48.835 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T17:57:48.835 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T17:57:48.835 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T17:57:48.836 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 2023-10-18T17:57:48.836 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T17:57:48.836 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2023-10-18T17:57:48.836 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T17:57:48.836 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T17:57:48.837 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T17:57:48.837 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T17:57:48.838 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2023-10-18T17:57:48.838 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-18T17:57:48.898 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-18T17:57:48.908 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T17:57:48.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:57:48.910 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-18T17:57:48.920 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T17:57:48.921 INFO:tasks.ceph:role: osd.7 2023-10-18T17:57:48.921 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T17:57:48.922 DEBUG:teuthology.orchestra.run.smithi179:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-18T17:57:49.405 INFO:teuthology.orchestra.run.smithi179.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T17:57:49.406 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T17:57:49.406 INFO:teuthology.orchestra.run.smithi179.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T17:57:49.406 INFO:teuthology.orchestra.run.smithi179.stdout: = reflink=1 2023-10-18T17:57:49.406 INFO:teuthology.orchestra.run.smithi179.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T17:57:49.407 INFO:teuthology.orchestra.run.smithi179.stdout: = sunit=0 swidth=0 blks 2023-10-18T17:57:49.407 INFO:teuthology.orchestra.run.smithi179.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T17:57:49.407 INFO:teuthology.orchestra.run.smithi179.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-18T17:57:49.407 INFO:teuthology.orchestra.run.smithi179.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T17:57:49.407 INFO:teuthology.orchestra.run.smithi179.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T17:57:49.408 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi179.front.sepia.ceph.com -o noatime 2023-10-18T17:57:49.409 DEBUG:teuthology.orchestra.run.smithi179:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-18T17:57:49.469 DEBUG:teuthology.orchestra.run.smithi179:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-18T17:57:49.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T17:57:49.482 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: /sbin/restorecon: command not found 2023-10-18T17:57:49.483 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:49.555 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:49.551+0000 7f00d375a080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-18T17:57:49.559 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:49.555+0000 7f00d375a080 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-18T17:57:49.559 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:49.555+0000 7f00d375a080 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-18T17:57:49.559 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:49.555+0000 7f00d375a080 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-18T17:57:49.559 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:49.555+0000 7f00d375a080 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-18T17:57:49.563 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:49.559+0000 7f00d375a080 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-18T17:57:52.546 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-18T17:57:52.561 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:52.644 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:52.639+0000 7f4ae2e91080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-18T17:57:52.645 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:52.639+0000 7f4ae2e91080 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-18T17:57:52.646 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:52.643+0000 7f4ae2e91080 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-18T17:57:52.646 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:52.643+0000 7f4ae2e91080 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-18T17:57:52.646 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:52.643+0000 7f4ae2e91080 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-18T17:57:52.646 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:52.643+0000 7f4ae2e91080 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-18T17:57:55.621 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-18T17:57:55.637 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:55.678 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:55.675+0000 7fe331663080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-18T17:57:55.679 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:55.675+0000 7fe331663080 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-18T17:57:55.679 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:55.675+0000 7fe331663080 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-18T17:57:55.679 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:55.675+0000 7fe331663080 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-18T17:57:55.679 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:55.675+0000 7fe331663080 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-18T17:57:55.680 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:55.675+0000 7fe331663080 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-18T17:57:56.251 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:57:56.259 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:57:58.648 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-18T17:57:58.666 DEBUG:teuthology.orchestra.run.smithi179:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:57:58.704 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:58.698+0000 7fb517249080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-18T17:57:58.705 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:58.698+0000 7fb517249080 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-18T17:57:58.705 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:58.698+0000 7fb517249080 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-18T17:57:58.705 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:58.698+0000 7fb517249080 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-18T17:57:58.705 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:58.698+0000 7fb517249080 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-18T17:57:58.706 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:57:58.702+0000 7fb517249080 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-18T17:58:01.700 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-18T17:58:01.717 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-18T17:58:01.717 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:01.718 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-18T17:58:01.755 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:01.755 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-18T17:58:01.833 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:01.833 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-18T17:58:01.909 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:01.909 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2023-10-18T17:58:01.985 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:01.986 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-18T17:58:02.062 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:02.062 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-18T17:58:02.137 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:02.138 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-18T17:58:02.213 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:02.213 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-18T17:58:02.293 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:02.293 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-18T17:58:02.309 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:02.309 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-18T17:58:02.368 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:02.368 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-18T17:58:02.378 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:02.379 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2023-10-18T17:58:02.392 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:02.393 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-18T17:58:02.406 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:02.406 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-18T17:58:02.464 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:02.465 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-18T17:58:02.474 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:02.475 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-18T17:58:02.490 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:02.491 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-18T17:58:02.512 INFO:tasks.ceph:Adding keys to all mons... 2023-10-18T17:58:02.513 DEBUG:teuthology.orchestra.run.smithi072:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-18T17:58:02.563 DEBUG:teuthology.orchestra.run.smithi179:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-18T17:58:02.577 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr.y] 2023-10-18T17:58:02.577 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCIHDBlfZllJxAAjYBaIuYZOAbq9dIq7VdMkw== 2023-10-18T17:58:02.578 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.a] 2023-10-18T17:58:02.578 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCIHDBlnUyOMRAAVguQuUf9L5lDP6HdaCK3Vw== 2023-10-18T17:58:02.578 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.c] 2023-10-18T17:58:02.578 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCJHDBlaL4MABAAvY7tSaaY1fRvSZi17Zmbwg== 2023-10-18T17:58:02.578 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.e] 2023-10-18T17:58:02.579 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCJHDBl6VXICRAABg3TTFMJ/iLyqSNcMzz3zA== 2023-10-18T17:58:02.579 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.0] 2023-10-18T17:58:02.579 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCMHDBlcrQ3NxAAdv6yakGLvNkw6M66JrlB0g== 2023-10-18T17:58:02.579 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.1] 2023-10-18T17:58:02.579 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCQHDBlO+DrFRAA2v7vGCRRPxHJ2qO++nYUHA== 2023-10-18T17:58:02.580 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.2] 2023-10-18T17:58:02.580 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCUHDBlgmGVBBAAb3cHwtRxUfr2oC3DZFUfGg== 2023-10-18T17:58:02.580 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.3] 2023-10-18T17:58:02.581 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCXHDBlXiA5LhAA8mQ+IlQWmS4wn/TNaeVH7g== 2023-10-18T17:58:02.581 INFO:teuthology.orchestra.run.smithi179.stdout:[mgr.x] 2023-10-18T17:58:02.581 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCIHDBlJ6z5KxAAjk/2ftX0ZqDBPZB6pOi+2g== 2023-10-18T17:58:02.581 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.b] 2023-10-18T17:58:02.581 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCJHDBlgJ21DhAAOOSHFxEZbaf7uSKxYuMwcQ== 2023-10-18T17:58:02.581 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.d] 2023-10-18T17:58:02.582 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCJHDBlX2e8FBAA859up16Ob03b1S8y7KYGBQ== 2023-10-18T17:58:02.582 INFO:teuthology.orchestra.run.smithi179.stdout:[mds.f] 2023-10-18T17:58:02.582 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCJHDBlKyAHGBAAexre+VupIhyOWPjK1iDmQQ== 2023-10-18T17:58:02.582 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.4] 2023-10-18T17:58:02.582 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCdHDBlkZctIRAA06Gb/l7RvpkmGhewmqJnIw== 2023-10-18T17:58:02.583 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.5] 2023-10-18T17:58:02.583 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCgHDBlFmR6JhAAux9FjjoAL5k9SDrwToR/JQ== 2023-10-18T17:58:02.583 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.6] 2023-10-18T17:58:02.583 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCjHDBl5mqBKBAAxxWYpV+ORV0AvwiK/j50sQ== 2023-10-18T17:58:02.583 INFO:teuthology.orchestra.run.smithi179.stdout:[osd.7] 2023-10-18T17:58:02.584 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCmHDBlY98FKhAAScCc9hbfIz3lxDEHL26eVA== 2023-10-18T17:58:02.584 INFO:teuthology.orchestra.run.smithi179.stdout:[client.0] 2023-10-18T17:58:02.584 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQCJHDBlpN+dHBAAZLEGcnwut9h70SrWlQ6AlA== 2023-10-18T17:58:02.586 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.y] 2023-10-18T17:58:02.586 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCIHDBlfZllJxAAjYBaIuYZOAbq9dIq7VdMkw== 2023-10-18T17:58:02.587 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.a] 2023-10-18T17:58:02.587 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCIHDBlnUyOMRAAVguQuUf9L5lDP6HdaCK3Vw== 2023-10-18T17:58:02.587 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.c] 2023-10-18T17:58:02.587 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCJHDBlaL4MABAAvY7tSaaY1fRvSZi17Zmbwg== 2023-10-18T17:58:02.587 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.e] 2023-10-18T17:58:02.588 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCJHDBl6VXICRAABg3TTFMJ/iLyqSNcMzz3zA== 2023-10-18T17:58:02.588 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.0] 2023-10-18T17:58:02.588 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCMHDBlcrQ3NxAAdv6yakGLvNkw6M66JrlB0g== 2023-10-18T17:58:02.588 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.1] 2023-10-18T17:58:02.588 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCQHDBlO+DrFRAA2v7vGCRRPxHJ2qO++nYUHA== 2023-10-18T17:58:02.588 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.2] 2023-10-18T17:58:02.589 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCUHDBlgmGVBBAAb3cHwtRxUfr2oC3DZFUfGg== 2023-10-18T17:58:02.589 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.3] 2023-10-18T17:58:02.589 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCXHDBlXiA5LhAA8mQ+IlQWmS4wn/TNaeVH7g== 2023-10-18T17:58:02.589 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.x] 2023-10-18T17:58:02.589 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCIHDBlJ6z5KxAAjk/2ftX0ZqDBPZB6pOi+2g== 2023-10-18T17:58:02.590 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.b] 2023-10-18T17:58:02.590 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCJHDBlgJ21DhAAOOSHFxEZbaf7uSKxYuMwcQ== 2023-10-18T17:58:02.590 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.d] 2023-10-18T17:58:02.591 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCJHDBlX2e8FBAA859up16Ob03b1S8y7KYGBQ== 2023-10-18T17:58:02.591 INFO:teuthology.orchestra.run.smithi072.stdout:[mds.f] 2023-10-18T17:58:02.591 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCJHDBlKyAHGBAAexre+VupIhyOWPjK1iDmQQ== 2023-10-18T17:58:02.591 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.4] 2023-10-18T17:58:02.591 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCdHDBlkZctIRAA06Gb/l7RvpkmGhewmqJnIw== 2023-10-18T17:58:02.592 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.5] 2023-10-18T17:58:02.592 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCgHDBlFmR6JhAAux9FjjoAL5k9SDrwToR/JQ== 2023-10-18T17:58:02.592 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.6] 2023-10-18T17:58:02.592 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCjHDBl5mqBKBAAxxWYpV+ORV0AvwiK/j50sQ== 2023-10-18T17:58:02.592 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.7] 2023-10-18T17:58:02.592 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCmHDBlY98FKhAAScCc9hbfIz3lxDEHL26eVA== 2023-10-18T17:58:02.593 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2023-10-18T17:58:02.593 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCJHDBlpN+dHBAAZLEGcnwut9h70SrWlQ6AlA== 2023-10-18T17:58:02.594 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T17:58:02.643 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T17:58:02.708 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:02.715 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:02.781 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:02.788 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:02.860 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:02.867 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:02.932 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:02.939 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.003 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.010 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.072 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.079 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.151 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.158 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.221 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T17:58:03.227 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T17:58:03.291 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:03.298 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:03.365 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:03.372 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:03.439 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:03.445 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T17:58:03.510 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.516 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.580 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.586 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.650 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.657 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.720 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.727 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T17:58:03.790 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T17:58:03.797 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T17:58:03.862 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-18T17:58:03.863 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-18T17:58:03.894 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T17:58:04.037 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-18T17:58:04.072 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-18T17:58:04.089 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T17:58:04.157 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-18T17:58:04.172 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-18T17:58:04.229 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T17:58:04.299 DEBUG:teuthology.orchestra.run.smithi179:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-18T17:58:04.313 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:58:04.321 DEBUG:teuthology.orchestra.run.smithi179:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-18T17:58:04.332 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-18T17:58:04.333 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-18T17:58:04.333 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-18T17:58:04.382 INFO:tasks.ceph.mon.a:Started 2023-10-18T17:58:04.383 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-18T17:58:04.383 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-18T17:58:04.386 INFO:tasks.ceph.mon.b:Started 2023-10-18T17:58:04.386 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-18T17:58:04.386 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-18T17:58:04.390 INFO:tasks.ceph.mon.c:Started 2023-10-18T17:58:04.390 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-18T17:58:04.391 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-18T17:58:04.391 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-18T17:58:04.396 INFO:tasks.ceph.mgr.y:Started 2023-10-18T17:58:04.397 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-18T17:58:04.397 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-18T17:58:04.400 INFO:tasks.ceph.mgr.x:Started 2023-10-18T17:58:04.400 INFO:tasks.ceph:Setting crush tunables to default 2023-10-18T17:58:04.400 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd crush tunables default 2023-10-18T17:58:04.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67eb69700 1 Processor -- start 2023-10-18T17:58:04.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67eb69700 1 -- start start 2023-10-18T17:58:04.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67eb69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 0x7fd67814a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:04.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67eb69700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd6780689e0 0x7fd6780661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:04.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67eb69700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6781500e0 con 0x7fd6780c69f0 2023-10-18T17:58:04.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67eb69700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6781503c0 con 0x7fd6780689e0 2023-10-18T17:58:04.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67eb69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781506a0 con 0x7fd678149d80 2023-10-18T17:58:04.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd677fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd6780689e0 0x7fd6780661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:04.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67d106700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd6780c69f0 0x7fd6781517d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:60040/0 (socket says 172.21.15.72:60040) 2023-10-18T17:58:04.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67d106700 1 -- 172.21.15.72:0/105251372 learned_addr learned my addr 172.21.15.72:0/105251372 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:04.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67c905700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 0x7fd67814a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:04.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd677fff700 1 -- 172.21.15.72:0/105251372 >> v2:172.21.15.179:3300/0 conn(0x7fd6780689e0 msgr2=0x7fd6780661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-18T17:58:04.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd677fff700 1 -- 172.21.15.72:0/105251372 >> v2:172.21.15.179:3300/0 conn(0x7fd6780689e0 msgr2=0x7fd6780661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T17:58:04.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd677fff700 1 --2- 172.21.15.72:0/105251372 >> v2:172.21.15.179:3300/0 conn(0x7fd6780689e0 0x7fd6780661c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-18T17:58:04.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd677fff700 1 --2- 172.21.15.72:0/105251372 >> v2:172.21.15.179:3300/0 conn(0x7fd6780689e0 0x7fd6780661c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T17:58:04.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67c905700 1 -- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 msgr2=0x7fd67814a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-18T17:58:04.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67c905700 1 -- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 msgr2=0x7fd67814a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-18T17:58:04.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67c905700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 0x7fd67814a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-18T17:58:04.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.547+0000 7fd67c905700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 0x7fd67814a160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T17:58:04.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593384446 0 0) 0x7fd6781500e0 con 0x7fd6780c69f0 2023-10-18T17:58:04.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6680033f0 con 0x7fd6780c69f0 2023-10-18T17:58:04.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1942267079 0 0) 0x7fd6680033f0 con 0x7fd6780c69f0 2023-10-18T17:58:04.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 msgr2=0x7fd67814a160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T17:58:04.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 0x7fd67814a160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:04.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 >> v2:172.21.15.179:3300/0 conn(0x7fd6780689e0 msgr2=0x7fd6780661c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T17:58:04.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 --2- 172.21.15.72:0/105251372 >> v2:172.21.15.179:3300/0 conn(0x7fd6780689e0 0x7fd6780661c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:04.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678150980 con 0x7fd6780c69f0 2023-10-18T17:58:04.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1723103133 0 0) 0x7fd670001810 con 0x7fd6780c69f0 2023-10-18T17:58:04.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 >> v1:172.21.15.72:6789/0 conn(0x7fd6780c69f0 legacy=0x7fd6781517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:04.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 0x7fd6680038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:04.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd668003ec0 0x7fd668004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:04.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd668005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:04.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781503c0 con 0x7fd668003ec0 2023-10-18T17:58:04.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781506a0 con 0x7fd668004b80 2023-10-18T17:58:04.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66800b990 con 0x7fd678149d80 2023-10-18T17:58:04.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.583+0000 7fd67d106700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd668005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:04.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd67c905700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 0x7fd6680038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:04.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd677fff700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd668003ec0 0x7fd668004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:04.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd67d106700 1 -- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 msgr2=0x7fd6680038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:04.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd67d106700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd678149d80 0x7fd6680038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:04.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd67d106700 1 -- 172.21.15.72:0/105251372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd668003ec0 msgr2=0x7fd668004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:04.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd67d106700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd668003ec0 0x7fd668004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:04.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd67d106700 1 -- 172.21.15.72:0/105251372 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680033f0 con 0x7fd668004b80 2023-10-18T17:58:04.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd67d106700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd668005000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fd6780c43f0 tx=0x7fd670007b40).ready entity=mon.1 client_cookie=31e053b135ac7a9f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:04.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67000ce60 con 0x7fd668004b80 2023-10-18T17:58:04.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd670004af0 con 0x7fd668004b80 2023-10-18T17:58:04.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.587+0000 7fd6777fe700 1 -- 172.21.15.72:0/105251372 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670004e90 con 0x7fd668004b80 2023-10-18T17:58:04.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.591+0000 7fd67eb69700 1 -- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 msgr2=0x7fd668005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:04.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.591+0000 7fd67eb69700 1 --2- 172.21.15.72:0/105251372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd668005000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fd6780c43f0 tx=0x7fd670007b40).stop 2023-10-18T17:58:04.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.591+0000 7fd67d106700 1 -- 172.21.15.72:0/105251372 reap_dead start 2023-10-18T17:58:04.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.591+0000 7fd67eb69700 1 -- 172.21.15.72:0/105251372 shutdown_connections 2023-10-18T17:58:04.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.591+0000 7fd67eb69700 1 -- 172.21.15.72:0/105251372 >> 172.21.15.72:0/105251372 conn(0x7fd67805dc20 msgr2=0x7fd67805e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:04.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.591+0000 7fd67eb69700 1 -- 172.21.15.72:0/105251372 shutdown_connections 2023-10-18T17:58:04.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.591+0000 7fd67eb69700 1 -- 172.21.15.72:0/105251372 wait complete. 2023-10-18T17:58:04.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 Processor -- start 2023-10-18T17:58:04.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 -- start start 2023-10-18T17:58:04.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd678161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:04.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6780bd780 0x7fd6780bbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:04.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6780bf850 0x7fd6780bfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:04.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780bd2f0 con 0x7fd6780bf850 2023-10-18T17:58:04.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780b45f0 con 0x7fd668004b80 2023-10-18T17:58:04.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780bc420 con 0x7fd6780bd780 2023-10-18T17:58:04.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67c905700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd678161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:04.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67d106700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6780bf850 0x7fd6780bfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:04.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67c905700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd678161b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:37814/0 (socket says 172.21.15.72:37814) 2023-10-18T17:58:04.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67c905700 1 -- 172.21.15.72:0/61547131 learned_addr learned my addr 172.21.15.72:0/61547131 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:04.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd677fff700 1 --2- 172.21.15.72:0/61547131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6780bd780 0x7fd6780bbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:04.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67c905700 1 -- 172.21.15.72:0/61547131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6780bd780 msgr2=0x7fd6780bbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:04.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67c905700 1 --2- 172.21.15.72:0/61547131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6780bd780 0x7fd6780bbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:04.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67c905700 1 -- 172.21.15.72:0/61547131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6780bf850 msgr2=0x7fd6780bfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:04.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67c905700 1 --2- 172.21.15.72:0/61547131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6780bf850 0x7fd6780bfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:04.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67c905700 1 -- 172.21.15.72:0/61547131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678162080 con 0x7fd668004b80 2023-10-18T17:58:04.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67d106700 1 --2- 172.21.15.72:0/61547131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6780bf850 0x7fd6780bfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:04.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67c905700 1 --2- 172.21.15.72:0/61547131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd678161b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd66c010950 tx=0x7fd66c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:04.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd674ff9700 1 -- 172.21.15.72:0/61547131 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00e670 con 0x7fd668004b80 2023-10-18T17:58:04.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd674ff9700 1 -- 172.21.15.72:0/61547131 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c00e7d0 con 0x7fd668004b80 2023-10-18T17:58:04.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 -- 172.21.15.72:0/61547131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6780bc6e0 con 0x7fd668004b80 2023-10-18T17:58:04.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd674ff9700 1 -- 172.21.15.72:0/61547131 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00eb50 con 0x7fd668004b80 2023-10-18T17:58:04.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd67eb69700 1 -- 172.21.15.72:0/61547131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6780bcca0 con 0x7fd668004b80 2023-10-18T17:58:04.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd674ff9700 1 -- 172.21.15.72:0/61547131 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fd66c005ae0 con 0x7fd668004b80 2023-10-18T17:58:04.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.595+0000 7fd674ff9700 1 -- 172.21.15.72:0/61547131 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fd66c014470 con 0x7fd668004b80 2023-10-18T17:58:04.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.599+0000 7fd67eb69700 1 -- 172.21.15.72:0/61547131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6780c5b30 con 0x7fd668004b80 2023-10-18T17:58:04.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.603+0000 7fd674ff9700 1 -- 172.21.15.72:0/61547131 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fd6780c5b30 con 0x7fd668004b80 2023-10-18T17:58:04.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.671+0000 7fd67eb69700 1 -- 172.21.15.72:0/61547131 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd67806c190 con 0x7fd668004b80 2023-10-18T17:58:04.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd674ff9700 1 -- 172.21.15.72:0/61547131 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd66c006020 con 0x7fd668004b80 2023-10-18T17:58:04.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd67eb69700 1 -- 172.21.15.72:0/61547131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 msgr2=0x7fd678161b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:04.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd67eb69700 1 --2- 172.21.15.72:0/61547131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd678161b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd66c010950 tx=0x7fd66c008040).stop 2023-10-18T17:58:04.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd67eb69700 1 -- 172.21.15.72:0/61547131 shutdown_connections 2023-10-18T17:58:04.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd67eb69700 1 --2- 172.21.15.72:0/61547131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6780bf850 0x7fd6780bfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:04.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd67eb69700 1 --2- 172.21.15.72:0/61547131 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6780bd780 0x7fd6780bbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:04.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd67eb69700 1 --2- 172.21.15.72:0/61547131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd668004b80 0x7fd678161b90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:04.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd67eb69700 1 -- 172.21.15.72:0/61547131 >> 172.21.15.72:0/61547131 conn(0x7fd67805dc20 msgr2=0x7fd67814f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:04.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd67eb69700 1 -- 172.21.15.72:0/61547131 shutdown_connections 2023-10-18T17:58:04.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:04.683+0000 7fd67eb69700 1 -- 172.21.15.72:0/61547131 wait complete. 2023-10-18T17:58:04.689 INFO:teuthology.orchestra.run.smithi072.stderr:adjusted tunables profile to default 2023-10-18T17:58:04.706 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-18T17:58:04.707 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:04.707 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-18T17:58:04.744 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:04.744 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-18T17:58:04.818 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:04.819 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-18T17:58:04.827 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:04.822+0000 7fca4ff82500 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T17:58:04.841 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:04.839+0000 7fbf7fb31500 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T17:58:04.878 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-10-18T17:58:04.878 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-18T17:58:04.888 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:04.882+0000 7fca4ff82500 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T17:58:04.930 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:04.927+0000 7fbf7fb31500 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T17:58:04.952 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:04.952 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-18T17:58:04.970 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:04.970 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-18T17:58:05.028 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:05.028 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-18T17:58:05.039 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2023-10-18T17:58:05.039 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-18T17:58:05.053 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new f23d4e3d-4f47-4c2b-a28d-4997475203b2 0 2023-10-18T17:58:05.159 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f040a99e700 1 Processor -- start 2023-10-18T17:58:05.159 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f040a99e700 1 -- start start 2023-10-18T17:58:05.161 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f040a99e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f0404146640 0x7f04040be4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.161 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f040a99e700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04040689c0 con 0x7f040406a1e0 2023-10-18T17:58:05.162 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f040a99e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04040bd700 con 0x7f0404146a20 2023-10-18T17:58:05.162 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f040a99e700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040bd9e0 con 0x7f0404146640 2023-10-18T17:58:05.162 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f04037fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0404146a20 0x7f0404069190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:53166/0 (socket says 172.21.15.179:53166) 2023-10-18T17:58:05.162 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f04037fe700 1 -- 172.21.15.179:0/3066303844 learned_addr learned my addr 172.21.15.179:0/3066303844 (peer_addr_for_me v1:172.21.15.179:0/0) 2023-10-18T17:58:05.163 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f0403fff700 1 --2- 172.21.15.179:0/3066303844 >> v2:172.21.15.72:3300/0 conn(0x7f0404146640 0x7f04040be4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.163 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020062340 0 0) 0x7f04040bd700 con 0x7f0404146a20 2023-10-18T17:58:05.163 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.154+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03ec004500 con 0x7f0404146a20 2023-10-18T17:58:05.171 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 594210808 0 0) 0x7f03ec004500 con 0x7f0404146a20 2023-10-18T17:58:05.171 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 >> v2:172.21.15.72:3300/0 conn(0x7f0404146640 msgr2=0x7f04040be4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:05.173 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 --2- 172.21.15.179:0/3066303844 >> v2:172.21.15.72:3300/0 conn(0x7f0404146640 0x7f04040be4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.174 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 >> v1:172.21.15.72:6789/0 conn(0x7f040406a1e0 legacy=0x7f0404156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.174 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04040bdcc0 con 0x7f0404146a20 2023-10-18T17:58:05.174 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1723103133 0 0) 0x7f03f8001470 con 0x7f0404146a20 2023-10-18T17:58:05.174 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 >> v1:172.21.15.179:6789/0 conn(0x7f0404146a20 legacy=0x7f0404069190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.175 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec0049f0 0x7f03ec004e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.175 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03ec0057a0 0x7f03ec005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.175 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03ec006110 0x7f03ec0033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.176 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04040bd9e0 con 0x7f03ec0049f0 2023-10-18T17:58:05.176 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec003a10 con 0x7f03ec006110 2023-10-18T17:58:05.176 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec003bd0 con 0x7f03ec0057a0 2023-10-18T17:58:05.177 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f04037fe700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03ec0057a0 0x7f03ec005c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.177 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0408f3b700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03ec006110 0x7f03ec0033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.177 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0403fff700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec0049f0 0x7f03ec004e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.177 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0408f3b700 1 -- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03ec0057a0 msgr2=0x7f03ec005c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:05.178 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0408f3b700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03ec0057a0 0x7f03ec005c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.178 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0408f3b700 1 -- 172.21.15.179:0/3066303844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec0049f0 msgr2=0x7f03ec004e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:05.178 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0408f3b700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec0049f0 0x7f03ec004e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.179 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0408f3b700 1 -- 172.21.15.179:0/3066303844 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec004500 con 0x7f03ec006110 2023-10-18T17:58:05.179 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0403fff700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec0049f0 0x7f03ec004e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-18T17:58:05.179 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0408f3b700 1 -- 172.21.15.179:0/3066303844 reap_dead start 2023-10-18T17:58:05.180 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0408f3b700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03ec006110 0x7f03ec0033f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f03fc001e30 tx=0x7f03fc007f30).ready entity=mon.1 client_cookie=f97fed1b95536e82 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:05.180 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc006a20 con 0x7f03ec006110 2023-10-18T17:58:05.180 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc006b80 con 0x7f03ec006110 2023-10-18T17:58:05.181 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f0402ffd700 1 -- 172.21.15.179:0/3066303844 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00c5d0 con 0x7f03ec006110 2023-10-18T17:58:05.181 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f040a99e700 1 -- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03ec006110 msgr2=0x7f03ec0033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.181 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f040a99e700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03ec006110 0x7f03ec0033f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f03fc001e30 tx=0x7f03fc007f30).stop 2023-10-18T17:58:05.182 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f040a99e700 1 -- 172.21.15.179:0/3066303844 shutdown_connections 2023-10-18T17:58:05.182 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f040a99e700 1 --2- 172.21.15.179:0/3066303844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03ec006110 0x7f03ec0033f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.183 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f040a99e700 1 -- 172.21.15.179:0/3066303844 >> 172.21.15.179:0/3066303844 conn(0x7f04040c3450 msgr2=0x7f04040bf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:05.183 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f040a99e700 1 -- 172.21.15.179:0/3066303844 shutdown_connections 2023-10-18T17:58:05.183 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f040a99e700 1 -- 172.21.15.179:0/3066303844 wait complete. 2023-10-18T17:58:05.184 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f040a99e700 1 Processor -- start 2023-10-18T17:58:05.184 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.158+0000 7f040a99e700 1 -- start start 2023-10-18T17:58:05.184 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f040a99e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec006110 0x7f04040b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.185 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f040a99e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04040c1ab0 0x7f0404151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.185 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f040a99e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0404152210 0x7f0404147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.185 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f040a99e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0404149350 con 0x7f03ec006110 2023-10-18T17:58:05.186 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f040a99e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04041644d0 con 0x7f04040c1ab0 2023-10-18T17:58:05.186 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f040a99e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0404147c70 con 0x7f0404152210 2023-10-18T17:58:05.186 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f04037fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04040c1ab0 0x7f0404151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.187 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f0408f3b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0404152210 0x7f0404147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.187 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f04037fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04040c1ab0 0x7f0404151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56810/0 (socket says 172.21.15.179:56810) 2023-10-18T17:58:05.187 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f04037fe700 1 -- 172.21.15.179:0/91668643 learned_addr learned my addr 172.21.15.179:0/91668643 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:05.188 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f0403fff700 1 --2- 172.21.15.179:0/91668643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec006110 0x7f04040b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.188 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f0408f3b700 1 -- 172.21.15.179:0/91668643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04040c1ab0 msgr2=0x7f0404151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.189 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f0408f3b700 1 --2- 172.21.15.179:0/91668643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04040c1ab0 0x7f0404151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.189 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f0408f3b700 1 -- 172.21.15.179:0/91668643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec006110 msgr2=0x7f04040b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.189 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f0408f3b700 1 --2- 172.21.15.179:0/91668643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec006110 0x7f04040b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.190 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f0408f3b700 1 -- 172.21.15.179:0/91668643 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04040bc8c0 con 0x7f0404152210 2023-10-18T17:58:05.190 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f0403fff700 1 --2- 172.21.15.179:0/91668643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec006110 0x7f04040b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-18T17:58:05.190 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f0408f3b700 1 --2- 172.21.15.179:0/91668643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0404152210 0x7f0404147f30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f0404067800 tx=0x7f03fc016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:05.190 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f03ebfff700 1 -- 172.21.15.179:0/91668643 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00c5d0 con 0x7f0404152210 2023-10-18T17:58:05.190 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f03ebfff700 1 -- 172.21.15.179:0/91668643 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc013bc0 con 0x7f0404152210 2023-10-18T17:58:05.191 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f040a99e700 1 -- 172.21.15.179:0/91668643 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0404163850 con 0x7f0404152210 2023-10-18T17:58:05.191 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f03ebfff700 1 -- 172.21.15.179:0/91668643 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc006a20 con 0x7f0404152210 2023-10-18T17:58:05.191 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f040a99e700 1 -- 172.21.15.179:0/91668643 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0404163e10 con 0x7f0404152210 2023-10-18T17:58:05.191 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f03ebfff700 1 -- 172.21.15.179:0/91668643 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f03fc01d8e0 con 0x7f0404152210 2023-10-18T17:58:05.192 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f03ebfff700 1 -- 172.21.15.179:0/91668643 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f03fc0277d0 con 0x7f0404152210 2023-10-18T17:58:05.192 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.162+0000 7f040a99e700 1 -- 172.21.15.179:0/91668643 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0404164610 con 0x7f0404152210 2023-10-18T17:58:05.193 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.166+0000 7f03ebfff700 1 -- 172.21.15.179:0/91668643 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f0404164610 con 0x7f0404152210 2023-10-18T17:58:05.245 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.238+0000 7f040a99e700 1 -- 172.21.15.179:0/91668643 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f23d4e3d-4f47-4c2b-a28d-4997475203b2", "id": 0} v 0) v1 -- 0x7f04040bdfa0 con 0x7f0404152210 2023-10-18T17:58:05.253 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f03ebfff700 1 -- 172.21.15.179:0/91668643 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f23d4e3d-4f47-4c2b-a28d-4997475203b2", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f03fc01ddc0 con 0x7f0404152210 2023-10-18T17:58:05.254 INFO:teuthology.orchestra.run.smithi179.stdout:0 2023-10-18T17:58:05.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f040a99e700 1 -- 172.21.15.179:0/91668643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0404152210 msgr2=0x7f0404147f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f040a99e700 1 --2- 172.21.15.179:0/91668643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0404152210 0x7f0404147f30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f0404067800 tx=0x7f03fc016000).stop 2023-10-18T17:58:05.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f040a99e700 1 -- 172.21.15.179:0/91668643 shutdown_connections 2023-10-18T17:58:05.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f040a99e700 1 --2- 172.21.15.179:0/91668643 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0404152210 0x7f0404147f30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f040a99e700 1 --2- 172.21.15.179:0/91668643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04040c1ab0 0x7f0404151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f040a99e700 1 --2- 172.21.15.179:0/91668643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03ec006110 0x7f04040b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f040a99e700 1 -- 172.21.15.179:0/91668643 >> 172.21.15.179:0/91668643 conn(0x7f04040c3450 msgr2=0x7f04040c0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:05.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f040a99e700 1 -- 172.21.15.179:0/91668643 shutdown_connections 2023-10-18T17:58:05.257 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.250+0000 7f040a99e700 1 -- 172.21.15.179:0/91668643 wait complete. 2023-10-18T17:58:05.268 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 214efe82-98a1-4870-9bcb-e3d6d3e42a28 1 2023-10-18T17:58:05.369 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.362+0000 7f6076ce9700 1 Processor -- start 2023-10-18T17:58:05.369 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.362+0000 7f6076ce9700 1 -- start start 2023-10-18T17:58:05.371 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f607005dd40 0x7f60700bc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.371 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f607014ae60 0x7f607014b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.371 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607014eab0 con 0x7f60700bbfd0 2023-10-18T17:58:05.371 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607014b7e0 con 0x7f607014ae60 2023-10-18T17:58:05.371 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607014dcb0 con 0x7f607005dd40 2023-10-18T17:58:05.372 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f607005dd40 0x7f60700bc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.372 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6074a85700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f60700bbfd0 0x7f60700bfb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:53180/0 (socket says 172.21.15.179:53180) 2023-10-18T17:58:05.372 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6074a85700 1 -- 172.21.15.179:0/4025456554 learned_addr learned my addr 172.21.15.179:0/4025456554 (peer_addr_for_me v1:172.21.15.179:0/0) 2023-10-18T17:58:05.374 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606ffff700 1 -- 172.21.15.179:0/4025456554 >> v2:172.21.15.72:3300/0 conn(0x7f607014ae60 msgr2=0x7f607014b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:05.374 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606ffff700 1 --2- 172.21.15.179:0/4025456554 >> v2:172.21.15.72:3300/0 conn(0x7f607014ae60 0x7f607014b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T17:58:05.374 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606ffff700 1 -- 172.21.15.179:0/4025456554 >> v1:172.21.15.179:6789/0 conn(0x7f60700bbfd0 legacy=0x7f60700bfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.374 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606ffff700 1 -- 172.21.15.179:0/4025456554 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60700bc8a0 con 0x7f607005dd40 2023-10-18T17:58:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606f7fe700 1 -- 172.21.15.179:0/4025456554 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317917701 0 0) 0x7f607014eab0 con 0x7f60700bbfd0 2023-10-18T17:58:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606ffff700 1 --2- 172.21.15.179:0/4025456554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f607005dd40 0x7f60700bc3b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f607005e480 tx=0x7f6064009060).ready entity=mon.1 client_cookie=a7d69c62561213ad server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606f7fe700 1 -- 172.21.15.179:0/4025456554 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400b610 con 0x7f607005dd40 2023-10-18T17:58:05.375 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606f7fe700 1 -- 172.21.15.179:0/4025456554 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606400b770 con 0x7f607005dd40 2023-10-18T17:58:05.377 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f606f7fe700 1 -- 172.21.15.179:0/4025456554 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400baf0 con 0x7f607005dd40 2023-10-18T17:58:05.377 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 -- 172.21.15.179:0/4025456554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f607005dd40 msgr2=0x7f60700bc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.377 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 --2- 172.21.15.179:0/4025456554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f607005dd40 0x7f60700bc3b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f607005e480 tx=0x7f6064009060).stop 2023-10-18T17:58:05.378 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 -- 172.21.15.179:0/4025456554 shutdown_connections 2023-10-18T17:58:05.378 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 --2- 172.21.15.179:0/4025456554 >> v2:172.21.15.72:3300/0 conn(0x7f607014ae60 0x7f607014b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T17:58:05.378 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 --2- 172.21.15.179:0/4025456554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f607005dd40 0x7f60700bc3b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.378 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 -- 172.21.15.179:0/4025456554 >> 172.21.15.179:0/4025456554 conn(0x7f60700c3450 msgr2=0x7f60700bf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:05.378 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 -- 172.21.15.179:0/4025456554 shutdown_connections 2023-10-18T17:58:05.379 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.366+0000 7f6076ce9700 1 -- 172.21.15.179:0/4025456554 wait complete. 2023-10-18T17:58:05.381 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 Processor -- start 2023-10-18T17:58:05.381 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 -- start start 2023-10-18T17:58:05.381 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f607005dd40 0x7f60701524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.382 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60700bbfd0 0x7f6070145330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.382 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f607014ae60 0x7f6070145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.382 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607014da80 con 0x7f607005dd40 2023-10-18T17:58:05.382 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607015ead0 con 0x7f607014ae60 2023-10-18T17:58:05.382 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6070153620 con 0x7f60700bbfd0 2023-10-18T17:58:05.383 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60700bbfd0 0x7f6070145330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.383 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60700bbfd0 0x7f6070145330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:33356/0 (socket says 172.21.15.179:33356) 2023-10-18T17:58:05.383 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606ffff700 1 -- 172.21.15.179:0/156798073 learned_addr learned my addr 172.21.15.179:0/156798073 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:05.383 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6075286700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f607014ae60 0x7f6070145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.383 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6074a85700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f607005dd40 0x7f60701524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.384 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606ffff700 1 -- 172.21.15.179:0/156798073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f607014ae60 msgr2=0x7f6070145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.384 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606ffff700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f607014ae60 0x7f6070145820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.384 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606ffff700 1 -- 172.21.15.179:0/156798073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f607005dd40 msgr2=0x7f60701524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.384 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606ffff700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f607005dd40 0x7f60701524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.384 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606ffff700 1 -- 172.21.15.179:0/156798073 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640092a0 con 0x7f60700bbfd0 2023-10-18T17:58:05.385 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6074a85700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f607005dd40 0x7f60701524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-18T17:58:05.385 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606ffff700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60700bbfd0 0x7f6070145330 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6064003070 tx=0x7f6064017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:05.385 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606cff9700 1 -- 172.21.15.179:0/156798073 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400b8e0 con 0x7f60700bbfd0 2023-10-18T17:58:05.385 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606cff9700 1 -- 172.21.15.179:0/156798073 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606400ba40 con 0x7f60700bbfd0 2023-10-18T17:58:05.385 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 -- 172.21.15.179:0/156798073 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607015e1a0 con 0x7f60700bbfd0 2023-10-18T17:58:05.386 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606cff9700 1 -- 172.21.15.179:0/156798073 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606401b620 con 0x7f60700bbfd0 2023-10-18T17:58:05.386 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f6076ce9700 1 -- 172.21.15.179:0/156798073 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607015e760 con 0x7f60700bbfd0 2023-10-18T17:58:05.386 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606cff9700 1 -- 172.21.15.179:0/156798073 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f606401bbf0 con 0x7f60700bbfd0 2023-10-18T17:58:05.386 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.370+0000 7f606cff9700 1 -- 172.21.15.179:0/156798073 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f6064023a60 con 0x7f60700bbfd0 2023-10-18T17:58:05.386 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.374+0000 7f6076ce9700 1 -- 172.21.15.179:0/156798073 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607006c190 con 0x7f60700bbfd0 2023-10-18T17:58:05.387 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.378+0000 7f606cff9700 1 -- 172.21.15.179:0/156798073 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f6064023ca0 con 0x7f60700bbfd0 2023-10-18T17:58:05.476 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.470+0000 7f6076ce9700 1 -- 172.21.15.179:0/156798073 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd new", "uuid": "214efe82-98a1-4870-9bcb-e3d6d3e42a28", "id": 1} v 0) v1 -- 0x7f607014dea0 con 0x7f60700bbfd0 2023-10-18T17:58:05.483 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f606cff9700 1 -- 172.21.15.179:0/156798073 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "214efe82-98a1-4870-9bcb-e3d6d3e42a28", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6064023e40 con 0x7f60700bbfd0 2023-10-18T17:58:05.484 INFO:teuthology.orchestra.run.smithi179.stdout:1 2023-10-18T17:58:05.485 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f6076ce9700 1 -- 172.21.15.179:0/156798073 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60700bbfd0 msgr2=0x7f6070145330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.485 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f6076ce9700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60700bbfd0 0x7f6070145330 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6064003070 tx=0x7f6064017000).stop 2023-10-18T17:58:05.485 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f6076ce9700 1 -- 172.21.15.179:0/156798073 shutdown_connections 2023-10-18T17:58:05.486 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f6076ce9700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f607014ae60 0x7f6070145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.486 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f6076ce9700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f60700bbfd0 0x7f6070145330 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.486 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f6076ce9700 1 --2- 172.21.15.179:0/156798073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f607005dd40 0x7f60701524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.486 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f6076ce9700 1 -- 172.21.15.179:0/156798073 >> 172.21.15.179:0/156798073 conn(0x7f60700c3450 msgr2=0x7f6070068840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:05.487 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f6076ce9700 1 -- 172.21.15.179:0/156798073 shutdown_connections 2023-10-18T17:58:05.487 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.478+0000 7f6076ce9700 1 -- 172.21.15.179:0/156798073 wait complete. 2023-10-18T17:58:05.497 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new e8b09618-8597-4bd0-a268-8e3f42c4544f 2 2023-10-18T17:58:05.537 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:05.534+0000 7fca4ff82500 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T17:58:05.608 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.602+0000 7f8764630700 1 Processor -- start 2023-10-18T17:58:05.608 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.602+0000 7f8764630700 1 -- start start 2023-10-18T17:58:05.610 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 0x7f875c062280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.610 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875c14eab0 con 0x7f875c05dd40 2023-10-18T17:58:05.610 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875c0bc840 con 0x7f875c0bbfd0 2023-10-18T17:58:05.610 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c14dcb0 con 0x7f875c0bc3b0 2023-10-18T17:58:05.611 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f87623cc700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f875c0bbfd0 0x7f875c0bfb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:53192/0 (socket says 172.21.15.179:53192) 2023-10-18T17:58:05.611 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f87623cc700 1 -- 172.21.15.179:0/376482142 learned_addr learned my addr 172.21.15.179:0/376482142 (peer_addr_for_me v1:172.21.15.179:0/0) 2023-10-18T17:58:05.611 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8762bcd700 1 --2- 172.21.15.179:0/376482142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 0x7f875c062280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.613 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f87613ca700 1 -- 172.21.15.179:0/376482142 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874674552 0 0) 0x7f875c0bc840 con 0x7f875c0bbfd0 2023-10-18T17:58:05.613 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f87613ca700 1 -- 172.21.15.179:0/376482142 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87480033f0 con 0x7f875c0bbfd0 2023-10-18T17:58:05.614 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8762bcd700 1 -- 172.21.15.179:0/376482142 >> v1:172.21.15.179:6789/0 conn(0x7f875c0bbfd0 legacy=0x7f875c0bfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.614 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8762bcd700 1 -- 172.21.15.179:0/376482142 >> v1:172.21.15.72:6789/0 conn(0x7f875c05dd40 legacy=0x7f875c14ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:05.614 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8762bcd700 1 -- 172.21.15.179:0/376482142 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c14dea0 con 0x7f875c0bc3b0 2023-10-18T17:58:05.614 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8762bcd700 1 --2- 172.21.15.179:0/376482142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 0x7f875c062280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f8754009e90 tx=0x7f8754009780).ready entity=mon.2 client_cookie=e20fbc1165ccc9ee server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:05.614 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f87613ca700 1 -- 172.21.15.179:0/376482142 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875400e070 con 0x7f875c0bc3b0 2023-10-18T17:58:05.615 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f87613ca700 1 -- 172.21.15.179:0/376482142 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875400bbb0 con 0x7f875c0bc3b0 2023-10-18T17:58:05.615 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f87613ca700 1 -- 172.21.15.179:0/376482142 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754012410 con 0x7f875c0bc3b0 2023-10-18T17:58:05.615 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 -- 172.21.15.179:0/376482142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 msgr2=0x7f875c062280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.615 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 --2- 172.21.15.179:0/376482142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 0x7f875c062280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f8754009e90 tx=0x7f8754009780).stop 2023-10-18T17:58:05.616 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 -- 172.21.15.179:0/376482142 shutdown_connections 2023-10-18T17:58:05.616 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 --2- 172.21.15.179:0/376482142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 0x7f875c062280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.616 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 -- 172.21.15.179:0/376482142 >> 172.21.15.179:0/376482142 conn(0x7f875c0c3450 msgr2=0x7f875c0bf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:05.616 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 -- 172.21.15.179:0/376482142 shutdown_connections 2023-10-18T17:58:05.617 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.606+0000 7f8764630700 1 -- 172.21.15.179:0/376482142 wait complete. 2023-10-18T17:58:05.617 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:05.615+0000 7fbf7fb31500 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T17:58:05.620 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 Processor -- start 2023-10-18T17:58:05.620 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 -- start start 2023-10-18T17:58:05.620 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f875c05dd40 0x7f875c14cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.620 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f875c0bbfd0 0x7f875c154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.620 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 0x7f875c15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.621 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c0c09a0 con 0x7f875c0bbfd0 2023-10-18T17:58:05.621 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c0c0b50 con 0x7f875c05dd40 2023-10-18T17:58:05.621 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c161f20 con 0x7f875c0bc3b0 2023-10-18T17:58:05.621 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87623cc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f875c05dd40 0x7f875c14cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.622 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87623cc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f875c05dd40 0x7f875c14cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56832/0 (socket says 172.21.15.179:56832) 2023-10-18T17:58:05.622 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87623cc700 1 -- 172.21.15.179:0/968474171 learned_addr learned my addr 172.21.15.179:0/968474171 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:05.622 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8762bcd700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 0x7f875c15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.622 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8761bcb700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f875c0bbfd0 0x7f875c154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.622 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87623cc700 1 -- 172.21.15.179:0/968474171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 msgr2=0x7f875c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.623 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87623cc700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 0x7f875c15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.623 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87623cc700 1 -- 172.21.15.179:0/968474171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f875c0bbfd0 msgr2=0x7f875c154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.623 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87623cc700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f875c0bbfd0 0x7f875c154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.623 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87623cc700 1 -- 172.21.15.179:0/968474171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8754009a20 con 0x7f875c05dd40 2023-10-18T17:58:05.623 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8761bcb700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f875c0bbfd0 0x7f875c154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-18T17:58:05.624 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87623cc700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f875c05dd40 0x7f875c14cf30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f875800c5f0 tx=0x7f8758007c80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:05.624 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87527fc700 1 -- 172.21.15.179:0/968474171 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87580072e0 con 0x7f875c05dd40 2023-10-18T17:58:05.624 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87527fc700 1 -- 172.21.15.179:0/968474171 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758007440 con 0x7f875c05dd40 2023-10-18T17:58:05.624 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 -- 172.21.15.179:0/968474171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875c0c0da0 con 0x7f875c05dd40 2023-10-18T17:58:05.624 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87527fc700 1 -- 172.21.15.179:0/968474171 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87580057a0 con 0x7f875c05dd40 2023-10-18T17:58:05.624 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f8764630700 1 -- 172.21.15.179:0/968474171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875c1615f0 con 0x7f875c05dd40 2023-10-18T17:58:05.625 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87527fc700 1 -- 172.21.15.179:0/968474171 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f8758013450 con 0x7f875c05dd40 2023-10-18T17:58:05.625 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.610+0000 7f87527fc700 1 -- 172.21.15.179:0/968474171 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f8758005e30 con 0x7f875c05dd40 2023-10-18T17:58:05.625 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.614+0000 7f8764630700 1 -- 172.21.15.179:0/968474171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875c06c190 con 0x7f875c05dd40 2023-10-18T17:58:05.626 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.614+0000 7f87527fc700 1 -- 172.21.15.179:0/968474171 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f8758018070 con 0x7f875c05dd40 2023-10-18T17:58:05.692 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.686+0000 7f8764630700 1 -- 172.21.15.179:0/968474171 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e8b09618-8597-4bd0-a268-8e3f42c4544f", "id": 2} v 0) v1 -- 0x7f875c14e180 con 0x7f875c05dd40 2023-10-18T17:58:05.700 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.694+0000 7f87527fc700 1 -- 172.21.15.179:0/968474171 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8b09618-8597-4bd0-a268-8e3f42c4544f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8758013700 con 0x7f875c05dd40 2023-10-18T17:58:05.700 INFO:teuthology.orchestra.run.smithi179.stdout:2 2023-10-18T17:58:05.702 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.694+0000 7f8764630700 1 -- 172.21.15.179:0/968474171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f875c05dd40 msgr2=0x7f875c14cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.702 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.694+0000 7f8764630700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f875c05dd40 0x7f875c14cf30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f875800c5f0 tx=0x7f8758007c80).stop 2023-10-18T17:58:05.702 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.694+0000 7f8764630700 1 -- 172.21.15.179:0/968474171 shutdown_connections 2023-10-18T17:58:05.703 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.694+0000 7f8764630700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f875c0bc3b0 0x7f875c15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.703 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.694+0000 7f8764630700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f875c0bbfd0 0x7f875c154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.703 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.694+0000 7f8764630700 1 --2- 172.21.15.179:0/968474171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f875c05dd40 0x7f875c14cf30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.703 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.694+0000 7f8764630700 1 -- 172.21.15.179:0/968474171 >> 172.21.15.179:0/968474171 conn(0x7f875c0c3450 msgr2=0x7f875c1512a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:05.703 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.698+0000 7f8764630700 1 -- 172.21.15.179:0/968474171 shutdown_connections 2023-10-18T17:58:05.704 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.698+0000 7f8764630700 1 -- 172.21.15.179:0/968474171 wait complete. 2023-10-18T17:58:05.714 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 711d83d6-b781-4ea7-b2e3-03fbf6f8c73c 3 2023-10-18T17:58:05.817 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.810+0000 7f6489bcb700 1 Processor -- start 2023-10-18T17:58:05.817 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.810+0000 7f6489bcb700 1 -- start start 2023-10-18T17:58:05.819 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f64841498b0 0x7f648414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.819 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6484149c90 0x7f64840645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.819 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648414d380 con 0x7f64840688e0 2023-10-18T17:58:05.819 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64840b66e0 con 0x7f64841498b0 2023-10-18T17:58:05.820 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6484062480 con 0x7f6484149c90 2023-10-18T17:58:05.820 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6483fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6484149c90 0x7f64840645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.820 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6483fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6484149c90 0x7f64840645a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56838/0 (socket says 172.21.15.179:56838) 2023-10-18T17:58:05.820 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6483fff700 1 -- 172.21.15.179:0/2777836950 learned_addr learned my addr 172.21.15.179:0/2777836950 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:05.820 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6483fff700 1 -- 172.21.15.179:0/2777836950 >> v2:172.21.15.72:3300/0 conn(0x7f64841498b0 msgr2=0x7f648414ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:05.820 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6483fff700 1 --2- 172.21.15.179:0/2777836950 >> v2:172.21.15.72:3300/0 conn(0x7f64841498b0 0x7f648414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T17:58:05.821 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6483fff700 1 -- 172.21.15.179:0/2777836950 >> v1:172.21.15.72:6789/0 conn(0x7f64840688e0 legacy=0x7f6484069840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:05.821 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6483fff700 1 -- 172.21.15.179:0/2777836950 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6484062760 con 0x7f6484149c90 2023-10-18T17:58:05.821 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6483fff700 1 --2- 172.21.15.179:0/2777836950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6484149c90 0x7f64840645a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f6470006fd0 tx=0x7f64700090f0).ready entity=mon.1 client_cookie=209a66a4ef287dca server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:05.821 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f64837fe700 1 -- 172.21.15.179:0/2777836950 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647000b610 con 0x7f6484149c90 2023-10-18T17:58:05.821 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f64837fe700 1 -- 172.21.15.179:0/2777836950 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647000b770 con 0x7f6484149c90 2023-10-18T17:58:05.821 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f64837fe700 1 -- 172.21.15.179:0/2777836950 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647000baf0 con 0x7f6484149c90 2023-10-18T17:58:05.823 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 -- 172.21.15.179:0/2777836950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6484149c90 msgr2=0x7f64840645a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.823 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 --2- 172.21.15.179:0/2777836950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6484149c90 0x7f64840645a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f6470006fd0 tx=0x7f64700090f0).stop 2023-10-18T17:58:05.823 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 -- 172.21.15.179:0/2777836950 shutdown_connections 2023-10-18T17:58:05.823 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 --2- 172.21.15.179:0/2777836950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6484149c90 0x7f64840645a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.824 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 --2- 172.21.15.179:0/2777836950 >> v2:172.21.15.72:3300/0 conn(0x7f64841498b0 0x7f648414ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T17:58:05.824 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 -- 172.21.15.179:0/2777836950 >> 172.21.15.179:0/2777836950 conn(0x7f64840c2a50 msgr2=0x7f64840c2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:05.824 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 -- 172.21.15.179:0/2777836950 shutdown_connections 2023-10-18T17:58:05.824 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.814+0000 7f6489bcb700 1 -- 172.21.15.179:0/2777836950 wait complete. 2023-10-18T17:58:05.825 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 Processor -- start 2023-10-18T17:58:05.825 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 -- start start 2023-10-18T17:58:05.827 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64840688e0 0x7f648413d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.827 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64841498b0 0x7f648414fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.827 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6484149c90 0x7f648413db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:05.828 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64840bb440 con 0x7f6484149c90 2023-10-18T17:58:05.828 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648413cfe0 con 0x7f64841498b0 2023-10-18T17:58:05.828 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6484143130 con 0x7f64840688e0 2023-10-18T17:58:05.828 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6488bc9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64840688e0 0x7f648413d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.828 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6483fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64841498b0 0x7f648414fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.829 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6488bc9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64840688e0 0x7f648413d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:33376/0 (socket says 172.21.15.179:33376) 2023-10-18T17:58:05.829 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6488bc9700 1 -- 172.21.15.179:0/2884505278 learned_addr learned my addr 172.21.15.179:0/2884505278 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:05.829 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f64893ca700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6484149c90 0x7f648413db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:05.829 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6488bc9700 1 -- 172.21.15.179:0/2884505278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64841498b0 msgr2=0x7f648414fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.829 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6488bc9700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64841498b0 0x7f648414fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.830 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6488bc9700 1 -- 172.21.15.179:0/2884505278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6484149c90 msgr2=0x7f648413db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.830 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6488bc9700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6484149c90 0x7f648413db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.830 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6488bc9700 1 -- 172.21.15.179:0/2884505278 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6470009370 con 0x7f64840688e0 2023-10-18T17:58:05.830 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f64893ca700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6484149c90 0x7f648413db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-18T17:58:05.830 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6483fff700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64841498b0 0x7f648414fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:05.830 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6488bc9700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64840688e0 0x7f648413d470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f647800b560 tx=0x7f64780090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:05.831 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6480ff9700 1 -- 172.21.15.179:0/2884505278 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478002320 con 0x7f64840688e0 2023-10-18T17:58:05.831 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6480ff9700 1 -- 172.21.15.179:0/2884505278 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64780024a0 con 0x7f64840688e0 2023-10-18T17:58:05.831 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6480ff9700 1 -- 172.21.15.179:0/2884505278 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478002860 con 0x7f64840688e0 2023-10-18T17:58:05.831 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 -- 172.21.15.179:0/2884505278 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648413d960 con 0x7f64840688e0 2023-10-18T17:58:05.831 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6489bcb700 1 -- 172.21.15.179:0/2884505278 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6484141d30 con 0x7f64840688e0 2023-10-18T17:58:05.832 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6480ff9700 1 -- 172.21.15.179:0/2884505278 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f647801c440 con 0x7f64840688e0 2023-10-18T17:58:05.832 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.818+0000 7f6480ff9700 1 -- 172.21.15.179:0/2884505278 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f64780128d0 con 0x7f64840688e0 2023-10-18T17:58:05.832 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.822+0000 7f6489bcb700 1 -- 172.21.15.179:0/2884505278 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6484046f00 con 0x7f64840688e0 2023-10-18T17:58:05.832 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.826+0000 7f6480ff9700 1 -- 172.21.15.179:0/2884505278 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f6478012bf0 con 0x7f64840688e0 2023-10-18T17:58:05.904 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.898+0000 7f6489bcb700 1 -- 172.21.15.179:0/2884505278 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd new", "uuid": "711d83d6-b781-4ea7-b2e3-03fbf6f8c73c", "id": 3} v 0) v1 -- 0x7f6484062a40 con 0x7f64840688e0 2023-10-18T17:58:05.911 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.906+0000 7f6480ff9700 1 -- 172.21.15.179:0/2884505278 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "711d83d6-b781-4ea7-b2e3-03fbf6f8c73c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6478012e10 con 0x7f64840688e0 2023-10-18T17:58:05.912 INFO:teuthology.orchestra.run.smithi179.stdout:3 2023-10-18T17:58:05.913 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.906+0000 7f6489bcb700 1 -- 172.21.15.179:0/2884505278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64840688e0 msgr2=0x7f648413d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:05.913 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.906+0000 7f6489bcb700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64840688e0 0x7f648413d470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f647800b560 tx=0x7f64780090f0).stop 2023-10-18T17:58:05.914 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.910+0000 7f6489bcb700 1 -- 172.21.15.179:0/2884505278 shutdown_connections 2023-10-18T17:58:05.914 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.910+0000 7f6489bcb700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6484149c90 0x7f648413db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.914 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.910+0000 7f6489bcb700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64841498b0 0x7f648414fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.915 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.910+0000 7f6489bcb700 1 --2- 172.21.15.179:0/2884505278 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f64840688e0 0x7f648413d470 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:05.915 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.910+0000 7f6489bcb700 1 -- 172.21.15.179:0/2884505278 >> 172.21.15.179:0/2884505278 conn(0x7f64840c2a50 msgr2=0x7f6484069840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:05.915 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.910+0000 7f6489bcb700 1 -- 172.21.15.179:0/2884505278 shutdown_connections 2023-10-18T17:58:05.915 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:05.910+0000 7f6489bcb700 1 -- 172.21.15.179:0/2884505278 wait complete. 2023-10-18T17:58:05.926 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39 4 2023-10-18T17:58:06.032 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.026+0000 7f6891148700 1 Processor -- start 2023-10-18T17:58:06.033 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.026+0000 7f6891148700 1 -- start start 2023-10-18T17:58:06.035 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.026+0000 7f6891148700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c0bed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.026+0000 7f6891148700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f688c0b65c0 0x7f688c066480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.026+0000 7f6891148700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688c0669e0 con 0x7f688c14d9f0 2023-10-18T17:58:06.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.026+0000 7f6891148700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688c065ff0 con 0x7f688c0b65c0 2023-10-18T17:58:06.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.026+0000 7f6891148700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c066c10 con 0x7f688c0be920 2023-10-18T17:58:06.036 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688bfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c0bed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688bfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c0bed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:33382/0 (socket says 172.21.15.179:33382) 2023-10-18T17:58:06.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688bfff700 1 -- 172.21.15.179:0/406713983 learned_addr learned my addr 172.21.15.179:0/406713983 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:06.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688b7fe700 1 --2- 172.21.15.179:0/406713983 >> v2:172.21.15.72:3300/0 conn(0x7f688c0b65c0 0x7f688c066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688bfff700 1 -- 172.21.15.179:0/406713983 >> v2:172.21.15.72:3300/0 conn(0x7f688c0b65c0 msgr2=0x7f688c066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:06.037 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688bfff700 1 --2- 172.21.15.179:0/406713983 >> v2:172.21.15.72:3300/0 conn(0x7f688c0b65c0 0x7f688c066480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.038 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688bfff700 1 -- 172.21.15.179:0/406713983 >> v1:172.21.15.72:6789/0 conn(0x7f688c14d9f0 legacy=0x7f688c0620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:06.038 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688bfff700 1 -- 172.21.15.179:0/406713983 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c066eb0 con 0x7f688c0be920 2023-10-18T17:58:06.038 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688bfff700 1 --2- 172.21.15.179:0/406713983 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c0bed00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f687c006fd0 tx=0x7f687c0090f0).ready entity=mon.2 client_cookie=3f63aa8153105241 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.038 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688affd700 1 -- 172.21.15.179:0/406713983 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c00b690 con 0x7f688c0be920 2023-10-18T17:58:06.039 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688affd700 1 -- 172.21.15.179:0/406713983 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c00b7f0 con 0x7f688c0be920 2023-10-18T17:58:06.039 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f688affd700 1 -- 172.21.15.179:0/406713983 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c00bb90 con 0x7f688c0be920 2023-10-18T17:58:06.039 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 -- 172.21.15.179:0/406713983 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 msgr2=0x7f688c0bed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.039 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 --2- 172.21.15.179:0/406713983 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c0bed00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f687c006fd0 tx=0x7f687c0090f0).stop 2023-10-18T17:58:06.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 -- 172.21.15.179:0/406713983 shutdown_connections 2023-10-18T17:58:06.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 --2- 172.21.15.179:0/406713983 >> v2:172.21.15.72:3300/0 conn(0x7f688c0b65c0 0x7f688c066480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 --2- 172.21.15.179:0/406713983 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c0bed00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 -- 172.21.15.179:0/406713983 >> 172.21.15.179:0/406713983 conn(0x7f688c05dc20 msgr2=0x7f688c05e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:06.040 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 -- 172.21.15.179:0/406713983 shutdown_connections 2023-10-18T17:58:06.041 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 -- 172.21.15.179:0/406713983 wait complete. 2023-10-18T17:58:06.043 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 Processor -- start 2023-10-18T17:58:06.043 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.030+0000 7f6891148700 1 -- start start 2023-10-18T17:58:06.043 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6891148700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f688c0b65c0 0x7f688c0c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.043 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6891148700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.043 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6891148700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c14d9f0 0x7f688c161d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.044 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6891148700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c14cd90 con 0x7f688c0b65c0 2023-10-18T17:58:06.044 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6891148700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0b7d90 con 0x7f688c14d9f0 2023-10-18T17:58:06.044 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6891148700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0b8070 con 0x7f688c0be920 2023-10-18T17:58:06.044 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f688b7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.044 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6890947700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c14d9f0 0x7f688c161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.044 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f688b7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c161830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:33390/0 (socket says 172.21.15.179:33390) 2023-10-18T17:58:06.045 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f688b7fe700 1 -- 172.21.15.179:0/3457206105 learned_addr learned my addr 172.21.15.179:0/3457206105 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:06.045 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6890947700 1 -- 172.21.15.179:0/3457206105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 msgr2=0x7f688c161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.045 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6890947700 1 --2- 172.21.15.179:0/3457206105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c161830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.045 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6890947700 1 -- 172.21.15.179:0/3457206105 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f688c0b65c0 msgr2=0x7f688c0c1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.045 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f688bfff700 1 --2- 172.21.15.179:0/3457206105 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f688c0b65c0 0x7f688c0c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.046 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6890947700 1 --2- 172.21.15.179:0/3457206105 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f688c0b65c0 0x7f688c0c1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.046 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6890947700 1 -- 172.21.15.179:0/3457206105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c009310 con 0x7f688c14d9f0 2023-10-18T17:58:06.046 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f688b7fe700 1 --2- 172.21.15.179:0/3457206105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:06.046 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6890947700 1 --2- 172.21.15.179:0/3457206105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c14d9f0 0x7f688c161d20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f688400b810 tx=0x7f68840091a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.046 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6873fff700 1 -- 172.21.15.179:0/3457206105 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400d4a0 con 0x7f688c14d9f0 2023-10-18T17:58:06.047 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6873fff700 1 -- 172.21.15.179:0/3457206105 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688400d620 con 0x7f688c14d9f0 2023-10-18T17:58:06.047 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6891148700 1 -- 172.21.15.179:0/3457206105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688c155cf0 con 0x7f688c14d9f0 2023-10-18T17:58:06.047 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6873fff700 1 -- 172.21.15.179:0/3457206105 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400d9e0 con 0x7f688c14d9f0 2023-10-18T17:58:06.047 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6891148700 1 -- 172.21.15.179:0/3457206105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688c0b6af0 con 0x7f688c14d9f0 2023-10-18T17:58:06.047 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6873fff700 1 -- 172.21.15.179:0/3457206105 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f688401b440 con 0x7f688c14d9f0 2023-10-18T17:58:06.048 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6873fff700 1 -- 172.21.15.179:0/3457206105 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f688401be20 con 0x7f688c14d9f0 2023-10-18T17:58:06.048 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.034+0000 7f6891148700 1 -- 172.21.15.179:0/3457206105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688c064de0 con 0x7f688c14d9f0 2023-10-18T17:58:06.048 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.038+0000 7f6873fff700 1 -- 172.21.15.179:0/3457206105 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f688c064de0 con 0x7f688c14d9f0 2023-10-18T17:58:06.117 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:06.110+0000 7fca4ff82500 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T17:58:06.118 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.114+0000 7f6891148700 1 -- 172.21.15.179:0/3457206105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39", "id": 4} v 0) v1 -- 0x7f688c046f00 con 0x7f688c14d9f0 2023-10-18T17:58:06.127 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6873fff700 1 -- 172.21.15.179:0/3457206105 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f68840290b0 con 0x7f688c14d9f0 2023-10-18T17:58:06.127 INFO:teuthology.orchestra.run.smithi179.stdout:4 2023-10-18T17:58:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6891148700 1 -- 172.21.15.179:0/3457206105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c14d9f0 msgr2=0x7f688c161d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6891148700 1 --2- 172.21.15.179:0/3457206105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c14d9f0 0x7f688c161d20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f688400b810 tx=0x7f68840091a0).stop 2023-10-18T17:58:06.129 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6891148700 1 -- 172.21.15.179:0/3457206105 shutdown_connections 2023-10-18T17:58:06.130 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6891148700 1 --2- 172.21.15.179:0/3457206105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c14d9f0 0x7f688c161d20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.130 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6891148700 1 --2- 172.21.15.179:0/3457206105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f688c0be920 0x7f688c161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.130 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6891148700 1 --2- 172.21.15.179:0/3457206105 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f688c0b65c0 0x7f688c0c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.130 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6891148700 1 -- 172.21.15.179:0/3457206105 >> 172.21.15.179:0/3457206105 conn(0x7f688c05dc20 msgr2=0x7f688c149cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:06.130 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6891148700 1 -- 172.21.15.179:0/3457206105 shutdown_connections 2023-10-18T17:58:06.131 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.122+0000 7f6891148700 1 -- 172.21.15.179:0/3457206105 wait complete. 2023-10-18T17:58:06.146 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 1557d884-e99b-4bf1-bed2-77d090311c62 5 2023-10-18T17:58:06.245 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.238+0000 7f9d0efe0700 1 Processor -- start 2023-10-18T17:58:06.246 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.238+0000 7f9d0efe0700 1 -- start start 2023-10-18T17:58:06.247 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d080b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.248 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d080bd9e0 con 0x7f9d080b6560 2023-10-18T17:58:06.248 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d08154010 con 0x7f9d08153b80 2023-10-18T17:58:06.248 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08069190 con 0x7f9d08069b00 2023-10-18T17:58:06.248 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0d7dd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d080b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.248 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0d7dd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d080b6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:33394/0 (socket says 172.21.15.179:33394) 2023-10-18T17:58:06.249 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0e7df700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9d08153b80 0x7f9d08150000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:53226/0 (socket says 172.21.15.179:53226) 2023-10-18T17:58:06.249 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0d7dd700 1 -- 172.21.15.179:0/4213194320 learned_addr learned my addr 172.21.15.179:0/4213194320 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:06.249 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0d7dd700 1 -- 172.21.15.179:0/4213194320 >> v1:172.21.15.179:6789/0 conn(0x7f9d08153b80 legacy=0x7f9d08150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.249 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0cfdc700 1 -- 172.21.15.179:0/4213194320 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813258775 0 0) 0x7f9d08154010 con 0x7f9d08153b80 2023-10-18T17:58:06.249 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0d7dd700 1 -- 172.21.15.179:0/4213194320 >> v1:172.21.15.72:6789/0 conn(0x7f9d080b6560 legacy=0x7f9d080bfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:06.250 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0d7dd700 1 -- 172.21.15.179:0/4213194320 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d08069470 con 0x7f9d08069b00 2023-10-18T17:58:06.250 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0d7dd700 1 --2- 172.21.15.179:0/4213194320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d080b6940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9cf8009830 tx=0x7f9cf80090f0).ready entity=mon.2 client_cookie=9a895c1d75da2316 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.250 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0cfdc700 1 -- 172.21.15.179:0/4213194320 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800b690 con 0x7f9d08069b00 2023-10-18T17:58:06.250 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0cfdc700 1 -- 172.21.15.179:0/4213194320 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf800b7f0 con 0x7f9d08069b00 2023-10-18T17:58:06.250 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0cfdc700 1 -- 172.21.15.179:0/4213194320 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf800bb70 con 0x7f9d08069b00 2023-10-18T17:58:06.254 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 -- 172.21.15.179:0/4213194320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 msgr2=0x7f9d080b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.254 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 --2- 172.21.15.179:0/4213194320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d080b6940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9cf8009830 tx=0x7f9cf80090f0).stop 2023-10-18T17:58:06.254 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 -- 172.21.15.179:0/4213194320 shutdown_connections 2023-10-18T17:58:06.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 --2- 172.21.15.179:0/4213194320 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d080b6940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 -- 172.21.15.179:0/4213194320 >> 172.21.15.179:0/4213194320 conn(0x7f9d0805dc20 msgr2=0x7f9d0805e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:06.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 -- 172.21.15.179:0/4213194320 shutdown_connections 2023-10-18T17:58:06.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.242+0000 7f9d0efe0700 1 -- 172.21.15.179:0/4213194320 wait complete. 2023-10-18T17:58:06.255 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 Processor -- start 2023-10-18T17:58:06.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 -- start start 2023-10-18T17:58:06.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d08152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d080b6560 0x7f9d080b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d08153b80 0x7f9d080b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080b3d70 con 0x7f9d08153b80 2023-10-18T17:58:06.256 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080b46a0 con 0x7f9d080b6560 2023-10-18T17:58:06.257 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d080bc8e0 con 0x7f9d08069b00 2023-10-18T17:58:06.257 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0dfde700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d08152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.257 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0d7dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d080b6560 0x7f9d080b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.257 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0dfde700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d08152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:33398/0 (socket says 172.21.15.179:33398) 2023-10-18T17:58:06.257 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0dfde700 1 -- 172.21.15.179:0/2839457389 learned_addr learned my addr 172.21.15.179:0/2839457389 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:06.257 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0dfde700 1 -- 172.21.15.179:0/2839457389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d080b6560 msgr2=0x7f9d080b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0dfde700 1 --2- 172.21.15.179:0/2839457389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d080b6560 0x7f9d080b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0dfde700 1 -- 172.21.15.179:0/2839457389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d08153b80 msgr2=0x7f9d080b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0dfde700 1 --2- 172.21.15.179:0/2839457389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d08153b80 0x7f9d080b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-18T17:58:06.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0dfde700 1 -- 172.21.15.179:0/2839457389 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8009370 con 0x7f9d08069b00 2023-10-18T17:58:06.258 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0d7dd700 1 --2- 172.21.15.179:0/2839457389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d080b6560 0x7f9d080b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:06.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0dfde700 1 --2- 172.21.15.179:0/2839457389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d08152110 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f9d0400b470 tx=0x7f9d04006f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9cfdffb700 1 -- 172.21.15.179:0/2839457389 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04012410 con 0x7f9d08069b00 2023-10-18T17:58:06.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9cfdffb700 1 -- 172.21.15.179:0/2839457389 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d040125b0 con 0x7f9d08069b00 2023-10-18T17:58:06.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9cfdffb700 1 -- 172.21.15.179:0/2839457389 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04012970 con 0x7f9d08069b00 2023-10-18T17:58:06.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 -- 172.21.15.179:0/2839457389 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d080b7030 con 0x7f9d08069b00 2023-10-18T17:58:06.259 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 -- 172.21.15.179:0/2839457389 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d080b75f0 con 0x7f9d08069b00 2023-10-18T17:58:06.260 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9cfdffb700 1 -- 172.21.15.179:0/2839457389 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f9d0401c440 con 0x7f9d08069b00 2023-10-18T17:58:06.260 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9cfdffb700 1 -- 172.21.15.179:0/2839457389 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f9d0400d980 con 0x7f9d08069b00 2023-10-18T17:58:06.260 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.246+0000 7f9d0efe0700 1 -- 172.21.15.179:0/2839457389 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d080bece0 con 0x7f9d08069b00 2023-10-18T17:58:06.260 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.250+0000 7f9cfdffb700 1 -- 172.21.15.179:0/2839457389 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f9d080bece0 con 0x7f9d08069b00 2023-10-18T17:58:06.270 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:06.266+0000 7fca4ff82500 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T17:58:06.326 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.322+0000 7f9d0efe0700 1 -- 172.21.15.179:0/2839457389 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1557d884-e99b-4bf1-bed2-77d090311c62", "id": 5} v 0) v1 -- 0x7f9d080b63c0 con 0x7f9d08069b00 2023-10-18T17:58:06.335 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9cfdffb700 1 -- 172.21.15.179:0/2839457389 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1557d884-e99b-4bf1-bed2-77d090311c62", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9d080b63c0 con 0x7f9d08069b00 2023-10-18T17:58:06.335 INFO:teuthology.orchestra.run.smithi179.stdout:5 2023-10-18T17:58:06.337 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9d0efe0700 1 -- 172.21.15.179:0/2839457389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 msgr2=0x7f9d08152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.337 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9d0efe0700 1 --2- 172.21.15.179:0/2839457389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d08152110 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f9d0400b470 tx=0x7f9d04006f90).stop 2023-10-18T17:58:06.338 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9d0efe0700 1 -- 172.21.15.179:0/2839457389 shutdown_connections 2023-10-18T17:58:06.338 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9d0efe0700 1 --2- 172.21.15.179:0/2839457389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9d08153b80 0x7f9d080b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-18T17:58:06.338 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9d0efe0700 1 --2- 172.21.15.179:0/2839457389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9d080b6560 0x7f9d080b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.338 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9d0efe0700 1 --2- 172.21.15.179:0/2839457389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9d08069b00 0x7f9d08152110 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.339 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9d0efe0700 1 -- 172.21.15.179:0/2839457389 >> 172.21.15.179:0/2839457389 conn(0x7f9d0805dc20 msgr2=0x7f9d08150580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:06.339 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9d0efe0700 1 -- 172.21.15.179:0/2839457389 shutdown_connections 2023-10-18T17:58:06.339 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.330+0000 7f9d0efe0700 1 -- 172.21.15.179:0/2839457389 wait complete. 2023-10-18T17:58:06.353 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 1425becf-644f-4f65-8172-6e39bf4dc94c 6 2023-10-18T17:58:06.381 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:06.378+0000 7fbf7fb31500 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T17:58:06.451 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aef2f6700 1 Processor -- start 2023-10-18T17:58:06.451 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aef2f6700 1 -- start start 2023-10-18T17:58:06.452 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aef2f6700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1ae80b9630 0x7f1ae8066300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.453 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aef2f6700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1ae80c0870 0x7f1ae814cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.453 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aef2f6700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae814d220 con 0x7f1ae80b9a10 2023-10-18T17:58:06.453 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aef2f6700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae814a880 con 0x7f1ae80c0870 2023-10-18T17:58:06.453 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aef2f6700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae814dcf0 con 0x7f1ae80b9630 2023-10-18T17:58:06.453 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aed092700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1ae80b9630 0x7f1ae8066300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.454 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aec891700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f1ae80b9a10 0x7f1ae80698e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.179:53236/0 (socket says 172.21.15.179:53236) 2023-10-18T17:58:06.454 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aec891700 1 -- 172.21.15.179:0/1170279065 learned_addr learned my addr 172.21.15.179:0/1170279065 (peer_addr_for_me v1:172.21.15.179:0/0) 2023-10-18T17:58:06.457 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aed893700 1 --2- 172.21.15.179:0/1170279065 >> v2:172.21.15.72:3300/0 conn(0x7f1ae80c0870 0x7f1ae814cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.458 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aed092700 1 -- 172.21.15.179:0/1170279065 >> v2:172.21.15.72:3300/0 conn(0x7f1ae80c0870 msgr2=0x7f1ae814cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:06.458 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aed092700 1 --2- 172.21.15.179:0/1170279065 >> v2:172.21.15.72:3300/0 conn(0x7f1ae80c0870 0x7f1ae814cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.458 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1adffff700 1 -- 172.21.15.179:0/1170279065 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697419587 0 0) 0x7f1ae814d220 con 0x7f1ae80b9a10 2023-10-18T17:58:06.458 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aed092700 1 -- 172.21.15.179:0/1170279065 >> v1:172.21.15.179:6789/0 conn(0x7f1ae80b9a10 legacy=0x7f1ae80698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.459 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aed092700 1 -- 172.21.15.179:0/1170279065 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae8069450 con 0x7f1ae80b9630 2023-10-18T17:58:06.459 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.446+0000 7f1aed092700 1 --2- 172.21.15.179:0/1170279065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9630 0x7f1ae8066300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f1ae805ea30 tx=0x7f1ae4009060).ready entity=mon.1 client_cookie=318dc697df9be611 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.459 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1adffff700 1 -- 172.21.15.179:0/1170279065 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400b580 con 0x7f1ae80b9630 2023-10-18T17:58:06.459 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1adffff700 1 -- 172.21.15.179:0/1170279065 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae400b6e0 con 0x7f1ae80b9630 2023-10-18T17:58:06.459 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1adffff700 1 -- 172.21.15.179:0/1170279065 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400ba60 con 0x7f1ae80b9630 2023-10-18T17:58:06.459 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- 172.21.15.179:0/1170279065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9630 msgr2=0x7f1ae8066300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.460 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 --2- 172.21.15.179:0/1170279065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9630 0x7f1ae8066300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f1ae805ea30 tx=0x7f1ae4009060).stop 2023-10-18T17:58:06.460 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- 172.21.15.179:0/1170279065 shutdown_connections 2023-10-18T17:58:06.460 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 --2- 172.21.15.179:0/1170279065 >> v2:172.21.15.72:3300/0 conn(0x7f1ae80c0870 0x7f1ae814cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.460 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 --2- 172.21.15.179:0/1170279065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9630 0x7f1ae8066300 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.460 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- 172.21.15.179:0/1170279065 >> 172.21.15.179:0/1170279065 conn(0x7f1ae805dc20 msgr2=0x7f1ae805e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:06.461 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- 172.21.15.179:0/1170279065 shutdown_connections 2023-10-18T17:58:06.461 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- 172.21.15.179:0/1170279065 wait complete. 2023-10-18T17:58:06.461 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 Processor -- start 2023-10-18T17:58:06.461 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- start start 2023-10-18T17:58:06.461 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ae80b9630 0x7f1ae8160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.461 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9a10 0x7f1ae81615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.462 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ae80c0870 0x7f1ae80b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.462 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae814bc30 con 0x7f1ae80c0870 2023-10-18T17:58:06.462 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81610b0 con 0x7f1ae80b9a10 2023-10-18T17:58:06.462 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae814ad40 con 0x7f1ae80b9630 2023-10-18T17:58:06.463 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aec891700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9a10 0x7f1ae81615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.463 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aed092700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ae80b9630 0x7f1ae8160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.463 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aec891700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9a10 0x7f1ae81615e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56878/0 (socket says 172.21.15.179:56878) 2023-10-18T17:58:06.463 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aec891700 1 -- 172.21.15.179:0/3079051974 learned_addr learned my addr 172.21.15.179:0/3079051974 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:06.463 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aed893700 1 --2- 172.21.15.179:0/3079051974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ae80c0870 0x7f1ae80b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aec891700 1 -- 172.21.15.179:0/3079051974 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ae80b9630 msgr2=0x7f1ae8160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aec891700 1 --2- 172.21.15.179:0/3079051974 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ae80b9630 0x7f1ae8160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aec891700 1 -- 172.21.15.179:0/3079051974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ae80c0870 msgr2=0x7f1ae80b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aec891700 1 --2- 172.21.15.179:0/3079051974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ae80c0870 0x7f1ae80b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aec891700 1 -- 172.21.15.179:0/3079051974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae40092a0 con 0x7f1ae80b9a10 2023-10-18T17:58:06.464 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aed092700 1 --2- 172.21.15.179:0/3079051974 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ae80b9630 0x7f1ae8160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:06.465 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aec891700 1 --2- 172.21.15.179:0/3079051974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9a10 0x7f1ae81615e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1ad800c4e0 tx=0x7f1ad8007c70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.465 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1add7fa700 1 -- 172.21.15.179:0/3079051974 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80072e0 con 0x7f1ae80b9a10 2023-10-18T17:58:06.465 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1add7fa700 1 -- 172.21.15.179:0/3079051974 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad8007440 con 0x7f1ae80b9a10 2023-10-18T17:58:06.465 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- 172.21.15.179:0/3079051974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae814b020 con 0x7f1ae80b9a10 2023-10-18T17:58:06.465 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1add7fa700 1 -- 172.21.15.179:0/3079051974 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80057a0 con 0x7f1ae80b9a10 2023-10-18T17:58:06.466 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1aef2f6700 1 -- 172.21.15.179:0/3079051974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae814b5e0 con 0x7f1ae80b9a10 2023-10-18T17:58:06.466 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.450+0000 7f1add7fa700 1 -- 172.21.15.179:0/3079051974 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f1ad8013450 con 0x7f1ae80b9a10 2023-10-18T17:58:06.466 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.454+0000 7f1add7fa700 1 -- 172.21.15.179:0/3079051974 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f1ad8005df0 con 0x7f1ae80b9a10 2023-10-18T17:58:06.467 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.454+0000 7f1aef2f6700 1 -- 172.21.15.179:0/3079051974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae814bde0 con 0x7f1ae80b9a10 2023-10-18T17:58:06.467 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.458+0000 7f1add7fa700 1 -- 172.21.15.179:0/3079051974 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f1ae814bde0 con 0x7f1ae80b9a10 2023-10-18T17:58:06.493 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:06.486+0000 7fca4ff82500 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T17:58:06.536 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.530+0000 7f1aef2f6700 1 -- 172.21.15.179:0/3079051974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1425becf-644f-4f65-8172-6e39bf4dc94c", "id": 6} v 0) v1 -- 0x7f1ae806c190 con 0x7f1ae80b9a10 2023-10-18T17:58:06.543 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1add7fa700 1 -- 172.21.15.179:0/3079051974 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1425becf-644f-4f65-8172-6e39bf4dc94c", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ad8013890 con 0x7f1ae80b9a10 2023-10-18T17:58:06.544 INFO:teuthology.orchestra.run.smithi179.stdout:6 2023-10-18T17:58:06.545 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1aef2f6700 1 -- 172.21.15.179:0/3079051974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9a10 msgr2=0x7f1ae81615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.545 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1aef2f6700 1 --2- 172.21.15.179:0/3079051974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9a10 0x7f1ae81615e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1ad800c4e0 tx=0x7f1ad8007c70).stop 2023-10-18T17:58:06.545 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1aef2f6700 1 -- 172.21.15.179:0/3079051974 shutdown_connections 2023-10-18T17:58:06.546 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1aef2f6700 1 --2- 172.21.15.179:0/3079051974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ae80c0870 0x7f1ae80b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.546 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1aef2f6700 1 --2- 172.21.15.179:0/3079051974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80b9a10 0x7f1ae81615e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.546 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1aef2f6700 1 --2- 172.21.15.179:0/3079051974 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1ae80b9630 0x7f1ae8160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.546 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1aef2f6700 1 -- 172.21.15.179:0/3079051974 >> 172.21.15.179:0/3079051974 conn(0x7f1ae805dc20 msgr2=0x7f1ae805e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:06.546 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1aef2f6700 1 -- 172.21.15.179:0/3079051974 shutdown_connections 2023-10-18T17:58:06.547 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.538+0000 7f1aef2f6700 1 -- 172.21.15.179:0/3079051974 wait complete. 2023-10-18T17:58:06.557 DEBUG:teuthology.orchestra.run.smithi179:> sudo ceph --cluster ceph osd new 8fdb41ca-c639-4d21-9338-d9416124f970 7 2023-10-18T17:58:06.599 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:06.594+0000 7fbf7fb31500 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T17:58:06.651 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:06.646+0000 7fca4ff82500 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T17:58:06.658 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f261b146700 1 Processor -- start 2023-10-18T17:58:06.659 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f261b146700 1 -- start start 2023-10-18T17:58:06.661 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f261b146700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f261414b720 0x7f26140bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.662 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f261b146700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261405dfb0 0x7f2614151890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.662 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f261b146700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2614151400 con 0x7f261414b340 2023-10-18T17:58:06.662 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f261b146700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261414de70 con 0x7f261414b720 2023-10-18T17:58:06.663 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f261b146700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140687d0 con 0x7f261405dfb0 2023-10-18T17:58:06.663 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26196e3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261405dfb0 0x7f2614151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.663 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26196e3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261405dfb0 0x7f2614151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.179:33420/0 (socket says 172.21.15.179:33420) 2023-10-18T17:58:06.664 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26196e3700 1 -- 172.21.15.179:0/590153395 learned_addr learned my addr 172.21.15.179:0/590153395 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:06.664 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f2613fff700 1 --2- 172.21.15.179:0/590153395 >> v2:172.21.15.179:3300/0 conn(0x7f261414b720 0x7f26140bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.665 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26196e3700 1 -- 172.21.15.179:0/590153395 >> v2:172.21.15.179:3300/0 conn(0x7f261414b720 msgr2=0x7f26140bd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:06.665 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26196e3700 1 --2- 172.21.15.179:0/590153395 >> v2:172.21.15.179:3300/0 conn(0x7f261414b720 0x7f26140bd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.665 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26196e3700 1 -- 172.21.15.179:0/590153395 >> v1:172.21.15.179:6789/0 conn(0x7f261414b340 legacy=0x7f26140b8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.666 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26137fe700 1 -- 172.21.15.179:0/590153395 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093687041 0 0) 0x7f2614151400 con 0x7f261414b340 2023-10-18T17:58:06.666 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26196e3700 1 -- 172.21.15.179:0/590153395 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140bf1f0 con 0x7f261405dfb0 2023-10-18T17:58:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f2613fff700 1 --2- 172.21.15.179:0/590153395 >> v2:172.21.15.179:3300/0 conn(0x7f261414b720 0x7f26140bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26196e3700 1 --2- 172.21.15.179:0/590153395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261405dfb0 0x7f2614151890 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f260c009f20 tx=0x7f260c009840).ready entity=mon.2 client_cookie=11d170ef535587a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.667 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26137fe700 1 -- 172.21.15.179:0/590153395 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00e070 con 0x7f261405dfb0 2023-10-18T17:58:06.668 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26137fe700 1 -- 172.21.15.179:0/590153395 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c00b9d0 con 0x7f261405dfb0 2023-10-18T17:58:06.668 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f26137fe700 1 -- 172.21.15.179:0/590153395 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00bd50 con 0x7f261405dfb0 2023-10-18T17:58:06.668 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f261b146700 1 -- 172.21.15.179:0/590153395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261405dfb0 msgr2=0x7f2614151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.669 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.654+0000 7f261b146700 1 --2- 172.21.15.179:0/590153395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261405dfb0 0x7f2614151890 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f260c009f20 tx=0x7f260c009840).stop 2023-10-18T17:58:06.669 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 -- 172.21.15.179:0/590153395 shutdown_connections 2023-10-18T17:58:06.669 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 --2- 172.21.15.179:0/590153395 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261405dfb0 0x7f2614151890 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.669 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 --2- 172.21.15.179:0/590153395 >> v2:172.21.15.179:3300/0 conn(0x7f261414b720 0x7f26140bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.670 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 -- 172.21.15.179:0/590153395 >> 172.21.15.179:0/590153395 conn(0x7f26140c2950 msgr2=0x7f26140c2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:06.670 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 -- 172.21.15.179:0/590153395 shutdown_connections 2023-10-18T17:58:06.670 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 -- 172.21.15.179:0/590153395 wait complete. 2023-10-18T17:58:06.675 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 Processor -- start 2023-10-18T17:58:06.675 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 -- start start 2023-10-18T17:58:06.675 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f261405dfb0 0x7f2614147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.676 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f261414b340 0x7f26141524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.676 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261414b720 0x7f2614145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.676 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614142040 con 0x7f261414b340 2023-10-18T17:58:06.676 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614141140 con 0x7f261405dfb0 2023-10-18T17:58:06.676 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f261b146700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261413fcd0 con 0x7f261414b720 2023-10-18T17:58:06.677 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f2618ee2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f261405dfb0 0x7f2614147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.677 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f2618ee2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f261405dfb0 0x7f2614147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56888/0 (socket says 172.21.15.179:56888) 2023-10-18T17:58:06.677 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f2618ee2700 1 -- 172.21.15.179:0/1325887525 learned_addr learned my addr 172.21.15.179:0/1325887525 (peer_addr_for_me v2:172.21.15.179:0/0) 2023-10-18T17:58:06.677 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f26196e3700 1 --2- 172.21.15.179:0/1325887525 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261414b720 0x7f2614145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.678 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f2618ee2700 1 -- 172.21.15.179:0/1325887525 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261414b720 msgr2=0x7f2614145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.678 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f2618ee2700 1 --2- 172.21.15.179:0/1325887525 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261414b720 0x7f2614145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.678 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f2618ee2700 1 -- 172.21.15.179:0/1325887525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f261414b340 msgr2=0x7f26141524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.678 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f2618ee2700 1 --2- 172.21.15.179:0/1325887525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f261414b340 0x7f26141524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-18T17:58:06.678 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f2618ee2700 1 -- 172.21.15.179:0/1325887525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc0092a0 con 0x7f261405dfb0 2023-10-18T17:58:06.679 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f26196e3700 1 --2- 172.21.15.179:0/1325887525 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261414b720 0x7f2614145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:06.679 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.658+0000 7f2618ee2700 1 --2- 172.21.15.179:0/1325887525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f261405dfb0 0x7f2614147e60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2604001eb0 tx=0x7f2604007c70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.679 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.662+0000 7f2610ff9700 1 -- 172.21.15.179:0/1325887525 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400f470 con 0x7f261405dfb0 2023-10-18T17:58:06.679 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.662+0000 7f2610ff9700 1 -- 172.21.15.179:0/1325887525 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400f5f0 con 0x7f261405dfb0 2023-10-18T17:58:06.680 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.662+0000 7f261b146700 1 -- 172.21.15.179:0/1325887525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c009ae0 con 0x7f261405dfb0 2023-10-18T17:58:06.680 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.662+0000 7f2610ff9700 1 -- 172.21.15.179:0/1325887525 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400f9b0 con 0x7f261405dfb0 2023-10-18T17:58:06.680 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.662+0000 7f261b146700 1 -- 172.21.15.179:0/1325887525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2614141e10 con 0x7f261405dfb0 2023-10-18T17:58:06.680 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.662+0000 7f2610ff9700 1 -- 172.21.15.179:0/1325887525 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f260401c440 con 0x7f261405dfb0 2023-10-18T17:58:06.681 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.662+0000 7f2610ff9700 1 -- 172.21.15.179:0/1325887525 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f2604012a90 con 0x7f261405dfb0 2023-10-18T17:58:06.681 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.662+0000 7f261b146700 1 -- 172.21.15.179:0/1325887525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261406c190 con 0x7f261405dfb0 2023-10-18T17:58:06.681 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.662+0000 7f2610ff9700 1 -- 172.21.15.179:0/1325887525 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f2604017070 con 0x7f261405dfb0 2023-10-18T17:58:06.702 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:06.698+0000 7fca4ff82500 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T17:58:06.744 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.738+0000 7f261b146700 1 -- 172.21.15.179:0/1325887525 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8fdb41ca-c639-4d21-9338-d9416124f970", "id": 7} v 0) v1 -- 0x7f2614141aa0 con 0x7f261405dfb0 2023-10-18T17:58:06.756 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f2610ff9700 1 -- 172.21.15.179:0/1325887525 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8fdb41ca-c639-4d21-9338-d9416124f970", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2604012ca0 con 0x7f261405dfb0 2023-10-18T17:58:06.756 INFO:teuthology.orchestra.run.smithi179.stdout:7 2023-10-18T17:58:06.757 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f261b146700 1 -- 172.21.15.179:0/1325887525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f261405dfb0 msgr2=0x7f2614147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.757 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f261b146700 1 --2- 172.21.15.179:0/1325887525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f261405dfb0 0x7f2614147e60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2604001eb0 tx=0x7f2604007c70).stop 2023-10-18T17:58:06.757 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f261b146700 1 -- 172.21.15.179:0/1325887525 shutdown_connections 2023-10-18T17:58:06.758 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f261b146700 1 --2- 172.21.15.179:0/1325887525 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f261414b720 0x7f2614145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.758 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f261b146700 1 --2- 172.21.15.179:0/1325887525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f261414b340 0x7f26141524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-10-18T17:58:06.758 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f261b146700 1 --2- 172.21.15.179:0/1325887525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f261405dfb0 0x7f2614147e60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.758 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f261b146700 1 -- 172.21.15.179:0/1325887525 >> 172.21.15.179:0/1325887525 conn(0x7f26140c2950 msgr2=0x7f2614142590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:06.758 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f261b146700 1 -- 172.21.15.179:0/1325887525 shutdown_connections 2023-10-18T17:58:06.759 INFO:teuthology.orchestra.run.smithi179.stderr:2023-10-18T17:58:06.750+0000 7f261b146700 1 -- 172.21.15.179:0/1325887525 wait complete. 2023-10-18T17:58:06.765 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:06.758+0000 7fca4ff82500 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T17:58:06.769 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-18T17:58:06.770 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-18T17:58:06.774 INFO:tasks.ceph.osd.0:Started 2023-10-18T17:58:06.774 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-18T17:58:06.775 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-18T17:58:06.778 INFO:tasks.ceph.osd.1:Started 2023-10-18T17:58:06.779 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-18T17:58:06.779 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-18T17:58:06.783 INFO:tasks.ceph.osd.2:Started 2023-10-18T17:58:06.783 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-18T17:58:06.784 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-18T17:58:06.790 INFO:tasks.ceph.osd.3:Started 2023-10-18T17:58:06.790 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-18T17:58:06.792 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-18T17:58:06.794 INFO:tasks.ceph.osd.4:Started 2023-10-18T17:58:06.794 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-18T17:58:06.794 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-18T17:58:06.796 INFO:tasks.ceph.osd.5:Started 2023-10-18T17:58:06.796 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-18T17:58:06.797 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-18T17:58:06.798 INFO:tasks.ceph.osd.6:Started 2023-10-18T17:58:06.799 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-18T17:58:06.799 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-18T17:58:06.801 INFO:tasks.ceph.osd.7:Started 2023-10-18T17:58:06.802 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:06.832 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:06.826+0000 7fca4ff82500 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T17:58:06.970 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:06.966+0000 7fbf7fb31500 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T17:58:06.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.970+0000 7f53c0769700 1 Processor -- start 2023-10-18T17:58:06.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.970+0000 7f53c0769700 1 -- start start 2023-10-18T17:58:06.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53c0769700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 0x7f53b814d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53c0769700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f53b80c3990 0x7f53b8066fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53c0769700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b80b33e0 con 0x7f53b80b35e0 2023-10-18T17:58:06.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53c0769700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b8066b30 con 0x7f53b80c3990 2023-10-18T17:58:06.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53c0769700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80bed40 con 0x7f53b814d090 2023-10-18T17:58:06.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bdd04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f53b80c3990 0x7f53b8066fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bdd04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f53b80c3990 0x7f53b8066fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39564/0 (socket says 172.21.15.72:39564) 2023-10-18T17:58:06.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bdd04700 1 -- 172.21.15.72:0/549084769 learned_addr learned my addr 172.21.15.72:0/549084769 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:06.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53be505700 1 --2- 172.21.15.72:0/549084769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 0x7f53b814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bdd04700 1 -- 172.21.15.72:0/549084769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 msgr2=0x7f53b814d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:06.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bdd04700 1 --2- 172.21.15.72:0/549084769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 0x7f53b814d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bdd04700 1 -- 172.21.15.72:0/549084769 >> v1:172.21.15.179:6789/0 conn(0x7f53b80b35e0 legacy=0x7f53b80620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bdd04700 1 -- 172.21.15.72:0/549084769 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b80befb0 con 0x7f53b80c3990 2023-10-18T17:58:06.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53be505700 1 --2- 172.21.15.72:0/549084769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 0x7f53b814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-18T17:58:06.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bdd04700 1 --2- 172.21.15.72:0/549084769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80c3990 0x7f53b8066fc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f53a8006fd0 tx=0x7f53a80090f0).ready entity=mon.0 client_cookie=9e4d61575b32e84 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bd503700 1 -- 172.21.15.72:0/549084769 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a800b600 con 0x7f53b80c3990 2023-10-18T17:58:06.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bd503700 1 -- 172.21.15.72:0/549084769 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a800b760 con 0x7f53b80c3990 2023-10-18T17:58:06.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.974+0000 7f53bd503700 1 -- 172.21.15.72:0/549084769 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a800bae0 con 0x7f53b80c3990 2023-10-18T17:58:06.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.978+0000 7f53c0769700 1 -- 172.21.15.72:0/549084769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80c3990 msgr2=0x7f53b8066fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.978+0000 7f53c0769700 1 --2- 172.21.15.72:0/549084769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80c3990 0x7f53b8066fc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f53a8006fd0 tx=0x7f53a80090f0).stop 2023-10-18T17:58:06.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.978+0000 7f53c0769700 1 -- 172.21.15.72:0/549084769 shutdown_connections 2023-10-18T17:58:06.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.978+0000 7f53c0769700 1 --2- 172.21.15.72:0/549084769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80c3990 0x7f53b8066fc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.978+0000 7f53c0769700 1 --2- 172.21.15.72:0/549084769 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 0x7f53b814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.978+0000 7f53c0769700 1 -- 172.21.15.72:0/549084769 >> 172.21.15.72:0/549084769 conn(0x7f53b805e6f0 msgr2=0x7f53b80c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:06.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.978+0000 7f53c0769700 1 -- 172.21.15.72:0/549084769 shutdown_connections 2023-10-18T17:58:06.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.978+0000 7f53c0769700 1 -- 172.21.15.72:0/549084769 wait complete. 2023-10-18T17:58:06.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.978+0000 7f53c0769700 1 Processor -- start 2023-10-18T17:58:06.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53c0769700 1 -- start start 2023-10-18T17:58:06.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53c0769700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80b35e0 0x7f53b814f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53c0769700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b80c3990 0x7f53b80c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53c0769700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 0x7f53b80b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:06.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53c0769700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80b92a0 con 0x7f53b80b35e0 2023-10-18T17:58:06.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53c0769700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80b7090 con 0x7f53b80c3990 2023-10-18T17:58:06.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53c0769700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80b9da0 con 0x7f53b814d090 2023-10-18T17:58:06.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53be505700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80b35e0 0x7f53b814f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53be505700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80b35e0 0x7f53b814f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39574/0 (socket says 172.21.15.72:39574) 2023-10-18T17:58:06.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53be505700 1 -- 172.21.15.72:0/604344308 learned_addr learned my addr 172.21.15.72:0/604344308 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:06.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53bdd04700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b80c3990 0x7f53b80c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53bed06700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 0x7f53b80b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:06.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53be505700 1 -- 172.21.15.72:0/604344308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 msgr2=0x7f53b80b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53be505700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 0x7f53b80b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53be505700 1 -- 172.21.15.72:0/604344308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b80c3990 msgr2=0x7f53b80c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:06.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53be505700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b80c3990 0x7f53b80c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:06.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53be505700 1 -- 172.21.15.72:0/604344308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a8009370 con 0x7f53b80b35e0 2023-10-18T17:58:06.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53bdd04700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b80c3990 0x7f53b80c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:06.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53be505700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80b35e0 0x7f53b814f880 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f53b400c880 tx=0x7f53b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:06.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53ae7fc700 1 -- 172.21.15.72:0/604344308 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4010580 con 0x7f53b80b35e0 2023-10-18T17:58:06.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53ae7fc700 1 -- 172.21.15.72:0/604344308 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b40106e0 con 0x7f53b80b35e0 2023-10-18T17:58:06.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53ae7fc700 1 -- 172.21.15.72:0/604344308 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4010a60 con 0x7f53b80b35e0 2023-10-18T17:58:06.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53c0769700 1 -- 172.21.15.72:0/604344308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b80b6ac0 con 0x7f53b80b35e0 2023-10-18T17:58:06.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53c0769700 1 -- 172.21.15.72:0/604344308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b814e900 con 0x7f53b80b35e0 2023-10-18T17:58:06.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.982+0000 7f53ae7fc700 1 -- 172.21.15.72:0/604344308 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f53b401b460 con 0x7f53b80b35e0 2023-10-18T17:58:06.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.986+0000 7f53ae7fc700 1 -- 172.21.15.72:0/604344308 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f53b4011b10 con 0x7f53b80b35e0 2023-10-18T17:58:06.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.986+0000 7f53c0769700 1 -- 172.21.15.72:0/604344308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b806c190 con 0x7f53b80b35e0 2023-10-18T17:58:06.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:06.990+0000 7f53ae7fc700 1 -- 172.21.15.72:0/604344308 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f53b4011cd0 con 0x7f53b80b35e0 2023-10-18T17:58:07.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.058+0000 7f53c0769700 1 -- 172.21.15.72:0/604344308 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53b80c3fe0 con 0x7f53b80b35e0 2023-10-18T17:58:07.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.058+0000 7f53ae7fc700 1 -- 172.21.15.72:0/604344308 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7f53b401b710 con 0x7f53b80b35e0 2023-10-18T17:58:07.064 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:07.064 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":10,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:06.744705+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:07.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.062+0000 7f53c0769700 1 -- 172.21.15.72:0/604344308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80b35e0 msgr2=0x7f53b814f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:07.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.062+0000 7f53c0769700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80b35e0 0x7f53b814f880 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f53b400c880 tx=0x7f53b4008040).stop 2023-10-18T17:58:07.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.062+0000 7f53c0769700 1 -- 172.21.15.72:0/604344308 shutdown_connections 2023-10-18T17:58:07.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.062+0000 7f53c0769700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53b814d090 0x7f53b80b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:07.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.062+0000 7f53c0769700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b80c3990 0x7f53b80c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:07.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.062+0000 7f53c0769700 1 --2- 172.21.15.72:0/604344308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53b80b35e0 0x7f53b814f880 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:07.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.062+0000 7f53c0769700 1 -- 172.21.15.72:0/604344308 >> 172.21.15.72:0/604344308 conn(0x7f53b805e6f0 msgr2=0x7f53b81515a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:07.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.062+0000 7f53c0769700 1 -- 172.21.15.72:0/604344308 shutdown_connections 2023-10-18T17:58:07.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:07.062+0000 7f53c0769700 1 -- 172.21.15.72:0/604344308 wait complete. 2023-10-18T17:58:07.086 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-18T17:58:07.087 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-18T17:58:07.131 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:07.126+0000 7fca4ff82500 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T17:58:07.179 INFO:tasks.ceph.osd.5.smithi179.stderr:2023-10-18T17:58:07.174+0000 7f6c1d92f080 -1 Falling back to public interface 2023-10-18T17:58:07.179 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:07.174+0000 7fbf7fb31500 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T17:58:07.255 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:07.250+0000 7fbf7fb31500 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T17:58:07.291 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:07.286+0000 7fca4ff82500 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T17:58:07.351 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:07.346+0000 7fbf7fb31500 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T17:58:07.390 INFO:tasks.ceph.osd.4.smithi179.stderr:2023-10-18T17:58:07.386+0000 7fae3e6f0080 -1 Falling back to public interface 2023-10-18T17:58:07.391 INFO:tasks.ceph.osd.7.smithi179.stderr:2023-10-18T17:58:07.386+0000 7f1241b02080 -1 Falling back to public interface 2023-10-18T17:58:07.407 INFO:tasks.ceph.osd.6.smithi179.stderr:2023-10-18T17:58:07.402+0000 7fd73b6ad080 -1 Falling back to public interface 2023-10-18T17:58:07.431 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:07.426+0000 7fbf7fb31500 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T17:58:07.447 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-10-18T17:58:07.442+0000 7f7984ced080 -1 Falling back to public interface 2023-10-18T17:58:07.462 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-10-18T17:58:07.458+0000 7f19876df080 -1 Falling back to public interface 2023-10-18T17:58:07.463 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-10-18T17:58:07.458+0000 7fbcc58da080 -1 Falling back to public interface 2023-10-18T17:58:07.463 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-10-18T17:58:07.458+0000 7f2f1474b080 -1 Falling back to public interface 2023-10-18T17:58:07.530 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:07.526+0000 7fca4ff82500 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T17:58:07.593 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:07.586+0000 7fca4ff82500 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T17:58:07.814 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:07.810+0000 7fbf7fb31500 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T17:58:07.820 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:07.814+0000 7fca4ff82500 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T17:58:07.875 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:07.870+0000 7fca4ff82500 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T17:58:07.893 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:08.004 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:07.998+0000 7fbf7fb31500 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T17:58:08.016 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:08.010+0000 7fca4ff82500 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T17:58:08.020 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b83039700 1 Processor -- start 2023-10-18T17:58:08.021 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b83039700 1 -- start start 2023-10-18T17:58:08.022 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b83039700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8b7c068a80 0x7f8b7c068e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:08.023 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b83039700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0be400 con 0x7f8b7c153b50 2023-10-18T17:58:08.023 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b83039700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c068690 con 0x7f8b7c151090 2023-10-18T17:58:08.023 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b83039700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c0a40 con 0x7f8b7c068a80 2023-10-18T17:58:08.023 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81836700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f8b7c153b50 0x7f8b7c0bff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:60054/0 (socket says 172.21.15.72:60054) 2023-10-18T17:58:08.023 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81836700 1 -- 172.21.15.72:0/2395877020 learned_addr learned my addr 172.21.15.72:0/2395877020 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:08.026 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b82037700 1 --2- 172.21.15.72:0/2395877020 >> v2:172.21.15.179:3300/0 conn(0x7f8b7c068a80 0x7f8b7c068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:08.026 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638085616 0 0) 0x7f8b7c0be400 con 0x7f8b7c153b50 2023-10-18T17:58:08.026 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b680033f0 con 0x7f8b7c153b50 2023-10-18T17:58:08.026 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686192745 0 0) 0x7f8b7c068690 con 0x7f8b7c151090 2023-10-18T17:58:08.027 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b7c0be400 con 0x7f8b7c151090 2023-10-18T17:58:08.027 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 70768855 0 0) 0x7f8b680033f0 con 0x7f8b7c153b50 2023-10-18T17:58:08.027 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 >> v2:172.21.15.179:3300/0 conn(0x7f8b7c068a80 msgr2=0x7f8b7c068e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:08.027 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 --2- 172.21.15.72:0/2395877020 >> v2:172.21.15.179:3300/0 conn(0x7f8b7c068a80 0x7f8b7c068e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:08.028 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 >> v1:172.21.15.179:6789/0 conn(0x7f8b7c151090 legacy=0x7f8b7c14cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:08.028 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0c0760 con 0x7f8b7c153b50 2023-10-18T17:58:08.028 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.018+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1723103133 0 0) 0x7f8b6c001470 con 0x7f8b7c153b50 2023-10-18T17:58:08.028 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 >> v1:172.21.15.72:6789/0 conn(0x7f8b7c153b50 legacy=0x7f8b7c0bff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:08.028 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b68003a70 0x7f8b68003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:08.029 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b68004690 0x7f8b68004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:08.029 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 0x7f8b6800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:08.029 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800be80 con 0x7f8b68003a70 2023-10-18T17:58:08.029 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800bfc0 con 0x7f8b68004690 2023-10-18T17:58:08.029 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800c180 con 0x7f8b68005000 2023-10-18T17:58:08.030 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82037700 1 --2- 172.21.15.72:0/2395877020 >> v2:172.21.15.179:3300/0 conn(0x7f8b7c068a80 0x7f8b7c068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:08.030 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81836700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b68004690 0x7f8b68004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:08.030 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82838700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 0x7f8b6800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:08.030 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82037700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b68003a70 0x7f8b68003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:08.031 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82838700 1 -- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b68004690 msgr2=0x7f8b68004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:08.031 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82838700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b68004690 0x7f8b68004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:08.031 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82838700 1 -- 172.21.15.72:0/2395877020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b68003a70 msgr2=0x7f8b68003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:08.031 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82838700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b68003a70 0x7f8b68003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:08.031 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82838700 1 -- 172.21.15.72:0/2395877020 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680033f0 con 0x7f8b68005000 2023-10-18T17:58:08.032 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82838700 1 -- 172.21.15.72:0/2395877020 reap_dead start 2023-10-18T17:58:08.032 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b82838700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 0x7f8b6800b8e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f8b74001f90 tx=0x7f8b74007f30).ready entity=mon.2 client_cookie=198fa50164f5d2ce server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:08.032 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74006a20 con 0x7f8b68005000 2023-10-18T17:58:08.032 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74006b80 con 0x7f8b68005000 2023-10-18T17:58:08.032 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b81035700 1 -- 172.21.15.72:0/2395877020 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7400c870 con 0x7f8b68005000 2023-10-18T17:58:08.035 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b83039700 1 -- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 msgr2=0x7f8b6800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:08.035 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b83039700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 0x7f8b6800b8e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f8b74001f90 tx=0x7f8b74007f30).stop 2023-10-18T17:58:08.036 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b83039700 1 -- 172.21.15.72:0/2395877020 shutdown_connections 2023-10-18T17:58:08.036 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b83039700 1 --2- 172.21.15.72:0/2395877020 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 0x7f8b6800b8e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:08.036 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b83039700 1 -- 172.21.15.72:0/2395877020 >> 172.21.15.72:0/2395877020 conn(0x7f8b7c06afa0 msgr2=0x7f8b7c06b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:08.036 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b83039700 1 -- 172.21.15.72:0/2395877020 shutdown_connections 2023-10-18T17:58:08.036 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.022+0000 7f8b83039700 1 -- 172.21.15.72:0/2395877020 wait complete. 2023-10-18T17:58:08.037 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 Processor -- start 2023-10-18T17:58:08.037 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 -- start start 2023-10-18T17:58:08.037 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 0x7f8b7c15fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:08.037 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0c1b30 0x7f8b7c0c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:08.037 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b7c1476c0 0x7f8b7c147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:08.038 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c3730 con 0x7f8b7c1476c0 2023-10-18T17:58:08.038 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c16a0 con 0x7f8b7c0c1b30 2023-10-18T17:58:08.038 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c2bb0 con 0x7f8b68005000 2023-10-18T17:58:08.038 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82838700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b7c1476c0 0x7f8b7c147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:08.038 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82037700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 0x7f8b7c15fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:08.039 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82838700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b7c1476c0 0x7f8b7c147b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39588/0 (socket says 172.21.15.72:39588) 2023-10-18T17:58:08.039 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82838700 1 -- 172.21.15.72:0/2893817577 learned_addr learned my addr 172.21.15.72:0/2893817577 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:08.039 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b81836700 1 --2- 172.21.15.72:0/2893817577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0c1b30 0x7f8b7c0c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:08.039 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82838700 1 -- 172.21.15.72:0/2893817577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 msgr2=0x7f8b7c15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:08.039 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82838700 1 --2- 172.21.15.72:0/2893817577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 0x7f8b7c15fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:08.040 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82838700 1 -- 172.21.15.72:0/2893817577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0c1b30 msgr2=0x7f8b7c0c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:08.040 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82838700 1 --2- 172.21.15.72:0/2893817577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0c1b30 0x7f8b7c0c1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:08.040 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82838700 1 -- 172.21.15.72:0/2893817577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0c2e90 con 0x7f8b7c1476c0 2023-10-18T17:58:08.040 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b82838700 1 --2- 172.21.15.72:0/2893817577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b7c1476c0 0x7f8b7c147b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f8b74009000 tx=0x7f8b74015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:08.041 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b727fc700 1 -- 172.21.15.72:0/2893817577 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7400c870 con 0x7f8b7c1476c0 2023-10-18T17:58:08.041 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b727fc700 1 -- 172.21.15.72:0/2893817577 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b740075e0 con 0x7f8b7c1476c0 2023-10-18T17:58:08.041 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 -- 172.21.15.72:0/2893817577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c0c3170 con 0x7f8b7c1476c0 2023-10-18T17:58:08.041 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b727fc700 1 -- 172.21.15.72:0/2893817577 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74006a20 con 0x7f8b7c1476c0 2023-10-18T17:58:08.041 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.026+0000 7f8b83039700 1 -- 172.21.15.72:0/2893817577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c0c2480 con 0x7f8b7c1476c0 2023-10-18T17:58:08.042 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.030+0000 7f8b727fc700 1 -- 172.21.15.72:0/2893817577 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f8b7401d890 con 0x7f8b7c1476c0 2023-10-18T17:58:08.042 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.030+0000 7f8b727fc700 1 -- 172.21.15.72:0/2893817577 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f8b7401db40 con 0x7f8b7c1476c0 2023-10-18T17:58:08.042 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.030+0000 7f8b83039700 1 -- 172.21.15.72:0/2893817577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c046f00 con 0x7f8b7c1476c0 2023-10-18T17:58:08.042 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.034+0000 7f8b727fc700 1 -- 172.21.15.72:0/2893817577 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f8b7402c060 con 0x7f8b7c1476c0 2023-10-18T17:58:08.103 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:08.098+0000 7fca4ff82500 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T17:58:08.104 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.098+0000 7f8b83039700 1 -- 172.21.15.72:0/2893817577 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b7c046f00 con 0x7f8b7c1476c0 2023-10-18T17:58:08.105 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b727fc700 1 -- 172.21.15.72:0/2893817577 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7f8b74022070 con 0x7f8b7c1476c0 2023-10-18T17:58:08.105 INFO:teuthology.misc.health.smithi072.stdout: 2023-10-18T17:58:08.106 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":10,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:06.744705+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:08.106 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b83039700 1 -- 172.21.15.72:0/2893817577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b7c1476c0 msgr2=0x7f8b7c147b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:08.106 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b83039700 1 --2- 172.21.15.72:0/2893817577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b7c1476c0 0x7f8b7c147b40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f8b74009000 tx=0x7f8b74015000).stop 2023-10-18T17:58:08.107 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b83039700 1 -- 172.21.15.72:0/2893817577 shutdown_connections 2023-10-18T17:58:08.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b83039700 1 --2- 172.21.15.72:0/2893817577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b7c1476c0 0x7f8b7c147b40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:08.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b83039700 1 --2- 172.21.15.72:0/2893817577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b7c0c1b30 0x7f8b7c0c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:08.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b83039700 1 --2- 172.21.15.72:0/2893817577 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8b68005000 0x7f8b7c15fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:08.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b83039700 1 -- 172.21.15.72:0/2893817577 >> 172.21.15.72:0/2893817577 conn(0x7f8b7c06afa0 msgr2=0x7f8b7c1499c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:08.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b83039700 1 -- 172.21.15.72:0/2893817577 shutdown_connections 2023-10-18T17:58:08.109 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:08.102+0000 7f8b83039700 1 -- 172.21.15.72:0/2893817577 wait complete. 2023-10-18T17:58:08.125 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-18T17:58:08.240 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:08.234+0000 7fca4ff82500 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T17:58:08.355 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:08.350+0000 7fbf7fb31500 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T17:58:08.429 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:08.422+0000 7fca4ff82500 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T17:58:08.445 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:08.442+0000 7fbf7fb31500 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T17:58:08.755 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:08.750+0000 7fbf7fb31500 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T17:58:08.842 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:08.838+0000 7fbf7fb31500 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T17:58:09.037 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:09.034+0000 7fbf7fb31500 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T17:58:09.170 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:09.166+0000 7fbf7fb31500 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T17:58:09.361 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:09.358+0000 7fbf7fb31500 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T17:58:09.475 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:09.470+0000 7fca39c63700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-18T17:58:09.475 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T17:58:09.470+0000 7fca39c63700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-18T17:58:09.615 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-10-18T17:58:09.610+0000 7fbf7fb31500 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T17:58:10.999 INFO:tasks.ceph.osd.5.smithi179.stderr:2023-10-18T17:58:10.994+0000 7f6c1d92f080 -1 osd.5 0 log_to_monitors {default=true} 2023-10-18T17:58:11.110 INFO:tasks.ceph.osd.4.smithi179.stderr:2023-10-18T17:58:11.106+0000 7fae3e6f0080 -1 osd.4 0 log_to_monitors {default=true} 2023-10-18T17:58:11.202 INFO:tasks.ceph.osd.6.smithi179.stderr:2023-10-18T17:58:11.198+0000 7fd73b6ad080 -1 osd.6 0 log_to_monitors {default=true} 2023-10-18T17:58:11.206 INFO:tasks.ceph.osd.7.smithi179.stderr:2023-10-18T17:58:11.202+0000 7f1241b02080 -1 osd.7 0 log_to_monitors {default=true} 2023-10-18T17:58:11.594 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-10-18T17:58:11.590+0000 7f7984ced080 -1 osd.2 0 log_to_monitors {default=true} 2023-10-18T17:58:11.618 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-10-18T17:58:11.614+0000 7fbcc58da080 -1 osd.1 0 log_to_monitors {default=true} 2023-10-18T17:58:11.740 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-10-18T17:58:11.738+0000 7f2f1474b080 -1 osd.3 0 log_to_monitors {default=true} 2023-10-18T17:58:11.741 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-10-18T17:58:11.738+0000 7f19876df080 -1 osd.0 0 log_to_monitors {default=true} 2023-10-18T17:58:12.554 INFO:tasks.ceph.osd.4.smithi179.stderr:2023-10-18T17:58:12.550+0000 7fae39e90700 -1 osd.4 0 waiting for initial osdmap 2023-10-18T17:58:12.555 INFO:tasks.ceph.osd.7.smithi179.stderr:2023-10-18T17:58:12.550+0000 7f123d2a2700 -1 osd.7 0 waiting for initial osdmap 2023-10-18T17:58:12.556 INFO:tasks.ceph.osd.6.smithi179.stderr:2023-10-18T17:58:12.550+0000 7fd736e4d700 -1 osd.6 0 waiting for initial osdmap 2023-10-18T17:58:12.556 INFO:tasks.ceph.osd.5.smithi179.stderr:2023-10-18T17:58:12.550+0000 7f6c190cf700 -1 osd.5 0 waiting for initial osdmap 2023-10-18T17:58:12.562 INFO:tasks.ceph.osd.5.smithi179.stderr:2023-10-18T17:58:12.558+0000 7f6c15e43700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T17:58:12.565 INFO:tasks.ceph.osd.4.smithi179.stderr:2023-10-18T17:58:12.558+0000 7fae36c04700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T17:58:12.565 INFO:tasks.ceph.osd.7.smithi179.stderr:2023-10-18T17:58:12.558+0000 7f123a016700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T17:58:12.566 INFO:tasks.ceph.osd.6.smithi179.stderr:2023-10-18T17:58:12.558+0000 7fd733bc1700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T17:58:13.553 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-10-18T17:58:13.550+0000 7f797fc1b700 -1 osd.2 0 waiting for initial osdmap 2023-10-18T17:58:13.554 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-10-18T17:58:13.550+0000 7fbcc0834700 -1 osd.1 0 waiting for initial osdmap 2023-10-18T17:58:13.554 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-10-18T17:58:13.550+0000 7f198265c700 -1 osd.0 0 waiting for initial osdmap 2023-10-18T17:58:13.555 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-10-18T17:58:13.550+0000 7f2f0feeb700 -1 osd.3 0 waiting for initial osdmap 2023-10-18T17:58:13.567 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-10-18T17:58:13.562+0000 7f797c98f700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T17:58:13.568 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-10-18T17:58:13.562+0000 7fbcbd5a8700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T17:58:13.568 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-10-18T17:58:13.562+0000 7f2f0cc5f700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T17:58:13.569 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-10-18T17:58:13.566+0000 7f197f3d0700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T17:58:14.932 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:15.091 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.086+0000 7fee82bb9700 1 Processor -- start 2023-10-18T17:58:15.091 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.086+0000 7fee82bb9700 1 -- start start 2023-10-18T17:58:15.093 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c06b4b0 0x7fee7c06b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.093 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fee7c14e1b0 0x7fee7c14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.093 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fee7c0b9500 0x7fee7c0b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.093 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7c14da80 con 0x7fee7c0b9500 2023-10-18T17:58:15.093 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7c0bfaa0 con 0x7fee7c14e1b0 2023-10-18T17:58:15.094 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c14dd20 con 0x7fee7c06b4b0 2023-10-18T17:58:15.095 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee823b8700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fee7c0b9500 0x7fee7c0b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.095 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee823b8700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fee7c0b9500 0x7fee7c0b99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49250/0 (socket says 172.21.15.72:49250) 2023-10-18T17:58:15.095 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee823b8700 1 -- 172.21.15.72:0/2196584374 learned_addr learned my addr 172.21.15.72:0/2196584374 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:15.096 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee81bb7700 1 --2- 172.21.15.72:0/2196584374 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c06b4b0 0x7fee7c06b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.096 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee813b6700 1 --2- 172.21.15.72:0/2196584374 >> v2:172.21.15.179:3300/0 conn(0x7fee7c14e1b0 0x7fee7c14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.096 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee823b8700 1 -- 172.21.15.72:0/2196584374 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c06b4b0 msgr2=0x7fee7c06b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:15.096 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee823b8700 1 --2- 172.21.15.72:0/2196584374 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c06b4b0 0x7fee7c06b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.096 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee823b8700 1 -- 172.21.15.72:0/2196584374 >> v2:172.21.15.179:3300/0 conn(0x7fee7c14e1b0 msgr2=0x7fee7c14e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:15.097 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee823b8700 1 --2- 172.21.15.72:0/2196584374 >> v2:172.21.15.179:3300/0 conn(0x7fee7c14e1b0 0x7fee7c14e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.097 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee823b8700 1 -- 172.21.15.72:0/2196584374 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c0c1fd0 con 0x7fee7c0b9500 2023-10-18T17:58:15.097 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee823b8700 1 --2- 172.21.15.72:0/2196584374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c0b9500 0x7fee7c0b99b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fee74009f20 tx=0x7fee74009840).ready entity=mon.0 client_cookie=63fabd90357d2a6c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:15.097 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee80bb5700 1 -- 172.21.15.72:0/2196584374 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7400e070 con 0x7fee7c0b9500 2023-10-18T17:58:15.097 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee80bb5700 1 -- 172.21.15.72:0/2196584374 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7400ba40 con 0x7fee7c0b9500 2023-10-18T17:58:15.097 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee80bb5700 1 -- 172.21.15.72:0/2196584374 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7400bdc0 con 0x7fee7c0b9500 2023-10-18T17:58:15.098 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 -- 172.21.15.72:0/2196584374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c0b9500 msgr2=0x7fee7c0b99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:15.099 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 --2- 172.21.15.72:0/2196584374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c0b9500 0x7fee7c0b99b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fee74009f20 tx=0x7fee74009840).stop 2023-10-18T17:58:15.099 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 -- 172.21.15.72:0/2196584374 shutdown_connections 2023-10-18T17:58:15.099 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 --2- 172.21.15.72:0/2196584374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c0b9500 0x7fee7c0b99b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.099 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 --2- 172.21.15.72:0/2196584374 >> v2:172.21.15.179:3300/0 conn(0x7fee7c14e1b0 0x7fee7c14e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.100 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 --2- 172.21.15.72:0/2196584374 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c06b4b0 0x7fee7c06b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.100 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.090+0000 7fee82bb9700 1 -- 172.21.15.72:0/2196584374 >> 172.21.15.72:0/2196584374 conn(0x7fee7c069d60 msgr2=0x7fee7c06a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:15.100 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 -- 172.21.15.72:0/2196584374 shutdown_connections 2023-10-18T17:58:15.100 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 -- 172.21.15.72:0/2196584374 wait complete. 2023-10-18T17:58:15.100 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 Processor -- start 2023-10-18T17:58:15.101 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 -- start start 2023-10-18T17:58:15.103 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c06b4b0 0x7fee7c14b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.103 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee7c0b9500 0x7fee7c0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.103 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c14e1b0 0x7fee7c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.103 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0b6340 con 0x7fee7c06b4b0 2023-10-18T17:58:15.103 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0b7820 con 0x7fee7c0b9500 2023-10-18T17:58:15.104 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.094+0000 7fee82bb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c1609b0 con 0x7fee7c14e1b0 2023-10-18T17:58:15.104 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee81bb7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c06b4b0 0x7fee7c14b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.104 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee813b6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee7c0b9500 0x7fee7c0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.104 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee823b8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c14e1b0 0x7fee7c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.104 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee813b6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee7c0b9500 0x7fee7c0b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:34736/0 (socket says 172.21.15.72:34736) 2023-10-18T17:58:15.105 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee813b6700 1 -- 172.21.15.72:0/142530338 learned_addr learned my addr 172.21.15.72:0/142530338 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:15.105 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee813b6700 1 -- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c14e1b0 msgr2=0x7fee7c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:15.105 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee813b6700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c14e1b0 0x7fee7c152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.105 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee813b6700 1 -- 172.21.15.72:0/142530338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c06b4b0 msgr2=0x7fee7c14b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:15.105 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee813b6700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c06b4b0 0x7fee7c14b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.105 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee813b6700 1 -- 172.21.15.72:0/142530338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee74009ae0 con 0x7fee7c0b9500 2023-10-18T17:58:15.106 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee81bb7700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c06b4b0 0x7fee7c14b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:15.106 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee813b6700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee7c0b9500 0x7fee7c0b5e50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fee6c00d8c0 tx=0x7fee6c008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:15.106 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee71ffb700 1 -- 172.21.15.72:0/142530338 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00f520 con 0x7fee7c0b9500 2023-10-18T17:58:15.106 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee71ffb700 1 -- 172.21.15.72:0/142530338 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c00f680 con 0x7fee7c0b9500 2023-10-18T17:58:15.106 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c1606d0 con 0x7fee7c0b9500 2023-10-18T17:58:15.107 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee71ffb700 1 -- 172.21.15.72:0/142530338 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00fa00 con 0x7fee7c0b9500 2023-10-18T17:58:15.107 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.098+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c14fcd0 con 0x7fee7c0b9500 2023-10-18T17:58:15.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.102+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c046f00 con 0x7fee7c0b9500 2023-10-18T17:58:15.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.102+0000 7fee71ffb700 1 -- 172.21.15.72:0/142530338 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fee6c013480 con 0x7fee7c0b9500 2023-10-18T17:58:15.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.102+0000 7fee71ffb700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fee6005a4e0 0x7fee6005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.102+0000 7fee71ffb700 1 -- 172.21.15.72:0/142530338 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7fee6c073ef0 con 0x7fee7c0b9500 2023-10-18T17:58:15.108 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.102+0000 7fee81bb7700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fee6005a4e0 0x7fee6005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.109 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.106+0000 7fee81bb7700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fee6005a4e0 0x7fee6005c9a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fee7800bfc0 tx=0x7fee78008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:15.117 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.114+0000 7fee71ffb700 1 -- 172.21.15.72:0/142530338 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fee6c047a80 con 0x7fee7c0b9500 2023-10-18T17:58:15.280 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.274+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee7c0b9ea0 con 0x7fee7c0b9500 2023-10-18T17:58:15.283 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.278+0000 7fee71ffb700 1 -- 172.21.15.72:0/142530338 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+12088 (secure 0 0 0) 0x7fee6c04b0a0 con 0x7fee7c0b9500 2023-10-18T17:58:15.283 INFO:teuthology.misc.health.smithi072.stdout: 2023-10-18T17:58:15.284 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":14,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:14.550597+0000","last_up_change":"2023-10-18T17:58:14.550597+0000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T17:58:13.627096+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":33232},{"type":"v1","addr":"172.21.15.72:6810","nonce":33232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6813","nonce":33232},{"type":"v1","addr":"172.21.15.72:6815","nonce":33232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":33232},{"type":"v1","addr":"172.21.15.72:6826","nonce":33232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":33232},{"type":"v1","addr":"172.21.15.72:6820","nonce":33232}]},"public_addr":"172.21.15.72:6810/33232","cluster_addr":"172.21.15.72:6815/33232","heartbeat_back_addr":"172.21.15.72:6826/33232","heartbeat_front_addr":"172.21.15.72:6820/33232","state":["exists","up"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6809","nonce":33229},{"type":"v1","addr":"172.21.15.72:6812","nonce":33229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":33229},{"type":"v1","addr":"172.21.15.72:6819","nonce":33229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":33229},{"type":"v1","addr":"172.21.15.72:6831","nonce":33229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":33229},{"type":"v1","addr":"172.21.15.72:6825","nonce":33229}]},"public_addr":"172.21.15.72:6812/33229","cluster_addr":"172.21.15.72:6819/33229","heartbeat_back_addr":"172.21.15.72:6831/33229","heartbeat_front_addr":"172.21.15.72:6825/33229","state":["exists","up"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":33231},{"type":"v1","addr":"172.21.15.72:6801","nonce":33231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":33231},{"type":"v1","addr":"172.21.15.72:6803","nonce":33231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":33231},{"type":"v1","addr":"172.21.15.72:6807","nonce":33231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":33231},{"type":"v1","addr":"172.21.15.72:6805","nonce":33231}]},"public_addr":"172.21.15.72:6801/33231","cluster_addr":"172.21.15.72:6803/33231","heartbeat_back_addr":"172.21.15.72:6807/33231","heartbeat_front_addr":"172.21.15.72:6805/33231","state":["exists","up"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":33228},{"type":"v1","addr":"172.21.15.72:6814","nonce":33228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":33228},{"type":"v1","addr":"172.21.15.72:6821","nonce":33228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":33228},{"type":"v1","addr":"172.21.15.72:6830","nonce":33228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":33228},{"type":"v1","addr":"172.21.15.72:6827","nonce":33228}]},"public_addr":"172.21.15.72:6814/33228","cluster_addr":"172.21.15.72:6821/33228","heartbeat_back_addr":"172.21.15.72:6830/33228","heartbeat_front_addr":"172.21.15.72:6827/33228","state":["exists","up"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":34034},{"type":"v1","addr":"172.21.15.179:6809","nonce":34034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":34034},{"type":"v1","addr":"172.21.15.179:6813","nonce":34034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":34034},{"type":"v1","addr":"172.21.15.179:6821","nonce":34034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":34034},{"type":"v1","addr":"172.21.15.179:6817","nonce":34034}]},"public_addr":"172.21.15.179:6809/34034","cluster_addr":"172.21.15.179:6813/34034","heartbeat_back_addr":"172.21.15.179:6821/34034","heartbeat_front_addr":"172.21.15.179:6817/34034","state":["exists","up"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":34044},{"type":"v1","addr":"172.21.15.179:6801","nonce":34044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":34044},{"type":"v1","addr":"172.21.15.179:6803","nonce":34044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":34044},{"type":"v1","addr":"172.21.15.179:6807","nonce":34044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":34044},{"type":"v1","addr":"172.21.15.179:6805","nonce":34044}]},"public_addr":"172.21.15.179:6801/34044","cluster_addr":"172.21.15.179:6803/34044","heartbeat_back_addr":"172.21.15.179:6807/34044","heartbeat_front_addr":"172.21.15.179:6805/34044","state":["exists","up"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":34036},{"type":"v1","addr":"172.21.15.179:6825","nonce":34036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":34036},{"type":"v1","addr":"172.21.15.179:6827","nonce":34036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":34036},{"type":"v1","addr":"172.21.15.179:6831","nonce":34036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":34036},{"type":"v1","addr":"172.21.15.179:6829","nonce":34036}]},"public_addr":"172.21.15.179:6825/34036","cluster_addr":"172.21.15.179:6827/34036","heartbeat_back_addr":"172.21.15.179:6831/34036","heartbeat_front_addr":"172.21.15.179:6829/34036","state":["exists","up"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":34035},{"type":"v1","addr":"172.21.15.179:6812","nonce":34035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":34035},{"type":"v1","addr":"172.21.15.179:6816","nonce":34035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":34035},{"type":"v1","addr":"172.21.15.179:6823","nonce":34035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":34035},{"type":"v1","addr":"172.21.15.179:6820","nonce":34035}]},"public_addr":"172.21.15.179:6812/34035","cluster_addr":"172.21.15.179:6816/34035","heartbeat_back_addr":"172.21.15.179:6823/34035","heartbeat_front_addr":"172.21.15.179:6820/34035","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.069244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:11.971339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.176699+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.197004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:15.286 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fee6005a4e0 msgr2=0x7fee6005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:15.286 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fee6005a4e0 0x7fee6005c9a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fee7800bfc0 tx=0x7fee78008040).stop 2023-10-18T17:58:15.286 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee7c0b9500 msgr2=0x7fee7c0b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:15.286 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee7c0b9500 0x7fee7c0b5e50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fee6c00d8c0 tx=0x7fee6c008040).stop 2023-10-18T17:58:15.287 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 shutdown_connections 2023-10-18T17:58:15.287 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fee6005a4e0 0x7fee6005c9a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.287 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fee7c14e1b0 0x7fee7c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.287 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee7c0b9500 0x7fee7c0b5e50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.287 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 --2- 172.21.15.72:0/142530338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee7c06b4b0 0x7fee7c14b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.287 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 >> 172.21.15.72:0/142530338 conn(0x7fee7c069d60 msgr2=0x7fee7c0670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:15.288 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 shutdown_connections 2023-10-18T17:58:15.288 INFO:teuthology.misc.health.smithi072.stderr:2023-10-18T17:58:15.282+0000 7fee82bb9700 1 -- 172.21.15.72:0/142530338 wait complete. 2023-10-18T17:58:15.304 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-18T17:58:15.305 INFO:tasks.ceph:Creating RBD pool 2023-10-18T17:58:15.305 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-18T17:58:15.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa5992e4700 1 Processor -- start 2023-10-18T17:58:15.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa5992e4700 1 -- start start 2023-10-18T17:58:15.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa5992e4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 0x7fa5940c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa5992e4700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa5940c27c0 0x7fa5940664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa5992e4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa594065670 0x7fa594065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa5992e4700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa594150df0 con 0x7fa5940c27c0 2023-10-18T17:58:15.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa5992e4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59414ffb0 con 0x7fa594065670 2023-10-18T17:58:15.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa5992e4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa594150270 con 0x7fa5940c23e0 2023-10-18T17:58:15.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa598ae3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa594065670 0x7fa594065ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa593fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 0x7fa5940c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa598ae3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa594065670 0x7fa594065ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:34752/0 (socket says 172.21.15.72:34752) 2023-10-18T17:58:15.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa598ae3700 1 -- 172.21.15.72:0/274459653 learned_addr learned my addr 172.21.15.72:0/274459653 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:15.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.462+0000 7fa5937fe700 1 --2- 172.21.15.72:0/274459653 >> v2:172.21.15.72:3300/0 conn(0x7fa5940c27c0 0x7fa5940664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa598ae3700 1 -- 172.21.15.72:0/274459653 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 msgr2=0x7fa5940c11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:15.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa598ae3700 1 --2- 172.21.15.72:0/274459653 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 0x7fa5940c11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa598ae3700 1 -- 172.21.15.72:0/274459653 >> v2:172.21.15.72:3300/0 conn(0x7fa5940c27c0 msgr2=0x7fa5940664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:15.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa598ae3700 1 --2- 172.21.15.72:0/274459653 >> v2:172.21.15.72:3300/0 conn(0x7fa5940c27c0 0x7fa5940664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa598ae3700 1 -- 172.21.15.72:0/274459653 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594150550 con 0x7fa594065670 2023-10-18T17:58:15.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5937fe700 1 --2- 172.21.15.72:0/274459653 >> v2:172.21.15.72:3300/0 conn(0x7fa5940c27c0 0x7fa5940664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:15.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa598ae3700 1 --2- 172.21.15.72:0/274459653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa594065670 0x7fa594065ac0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa594064bd0 tx=0x7fa58c009840).ready entity=mon.1 client_cookie=651c76d04b087815 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:15.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa592ffd700 1 -- 172.21.15.72:0/274459653 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00e070 con 0x7fa594065670 2023-10-18T17:58:15.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa592ffd700 1 -- 172.21.15.72:0/274459653 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00ba20 con 0x7fa594065670 2023-10-18T17:58:15.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa592ffd700 1 -- 172.21.15.72:0/274459653 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00bda0 con 0x7fa594065670 2023-10-18T17:58:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5992e4700 1 -- 172.21.15.72:0/274459653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa594065670 msgr2=0x7fa594065ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5992e4700 1 --2- 172.21.15.72:0/274459653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa594065670 0x7fa594065ac0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa594064bd0 tx=0x7fa58c009840).stop 2023-10-18T17:58:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5992e4700 1 -- 172.21.15.72:0/274459653 shutdown_connections 2023-10-18T17:58:15.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5992e4700 1 --2- 172.21.15.72:0/274459653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa594065670 0x7fa594065ac0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5992e4700 1 --2- 172.21.15.72:0/274459653 >> v2:172.21.15.72:3300/0 conn(0x7fa5940c27c0 0x7fa5940664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5992e4700 1 --2- 172.21.15.72:0/274459653 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 0x7fa5940c11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5992e4700 1 -- 172.21.15.72:0/274459653 >> 172.21.15.72:0/274459653 conn(0x7fa5940620d0 msgr2=0x7fa5940624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:15.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5992e4700 1 -- 172.21.15.72:0/274459653 shutdown_connections 2023-10-18T17:58:15.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.466+0000 7fa5992e4700 1 -- 172.21.15.72:0/274459653 wait complete. 2023-10-18T17:58:15.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa5992e4700 1 Processor -- start 2023-10-18T17:58:15.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa5992e4700 1 -- start start 2023-10-18T17:58:15.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa5992e4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa594065670 0x7fa5940bea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa5992e4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 0x7fa59414a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa5992e4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5940c27c0 0x7fa594146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa5992e4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594147000 con 0x7fa5940c27c0 2023-10-18T17:58:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa5992e4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940bce60 con 0x7fa594065670 2023-10-18T17:58:15.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa5992e4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940bd120 con 0x7fa5940c23e0 2023-10-18T17:58:15.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa598ae3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5940c27c0 0x7fa594146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa598ae3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5940c27c0 0x7fa594146180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49282/0 (socket says 172.21.15.72:49282) 2023-10-18T17:58:15.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa598ae3700 1 -- 172.21.15.72:0/2542488571 learned_addr learned my addr 172.21.15.72:0/2542488571 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:15.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa593fff700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa594065670 0x7fa5940bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.470+0000 7fa5937fe700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 0x7fa59414a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa598ae3700 1 -- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 msgr2=0x7fa59414a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:15.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa598ae3700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 0x7fa59414a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa598ae3700 1 -- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa594065670 msgr2=0x7fa5940bea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:15.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa598ae3700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa594065670 0x7fa5940bea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:15.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa598ae3700 1 -- 172.21.15.72:0/2542488571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c009ae0 con 0x7fa5940c27c0 2023-10-18T17:58:15.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa598ae3700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5940c27c0 0x7fa594146180 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fa5941500f0 tx=0x7fa5940bdb30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:15.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa57bfff700 1 -- 172.21.15.72:0/2542488571 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0032f0 con 0x7fa5940c27c0 2023-10-18T17:58:15.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa57bfff700 1 -- 172.21.15.72:0/2542488571 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c0092e0 con 0x7fa5940c27c0 2023-10-18T17:58:15.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa57bfff700 1 -- 172.21.15.72:0/2542488571 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c01e6a0 con 0x7fa5940c27c0 2023-10-18T17:58:15.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59414f650 con 0x7fa5940c27c0 2023-10-18T17:58:15.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59414fc10 con 0x7fa5940c27c0 2023-10-18T17:58:15.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.474+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa594147170 con 0x7fa5940c27c0 2023-10-18T17:58:15.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.478+0000 7fa57bfff700 1 -- 172.21.15.72:0/2542488571 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa58c003040 con 0x7fa5940c27c0 2023-10-18T17:58:15.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.478+0000 7fa57bfff700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa58005a540 0x7fa58005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:15.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.478+0000 7fa57bfff700 1 -- 172.21.15.72:0/2542488571 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7fa58c075600 con 0x7fa5940c27c0 2023-10-18T17:58:15.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.478+0000 7fa593fff700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa58005a540 0x7fa58005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:15.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.478+0000 7fa593fff700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa58005a540 0x7fa58005ca00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa584009c30 tx=0x7fa584009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:15.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.486+0000 7fa57bfff700 1 -- 172.21.15.72:0/2542488571 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fa594147170 con 0x7fa5940c27c0 2023-10-18T17:58:15.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:15.662+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fa594147140 con 0x7fa5940c27c0 2023-10-18T17:58:16.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.590+0000 7fa57bfff700 1 -- 172.21.15.72:0/2542488571 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7fa58c023030 con 0x7fa5940c27c0 2023-10-18T17:58:16.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa58005a540 msgr2=0x7fa58005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:16.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa58005a540 0x7fa58005ca00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa584009c30 tx=0x7fa584009060).stop 2023-10-18T17:58:16.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5940c27c0 msgr2=0x7fa594146180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:16.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5940c27c0 0x7fa594146180 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fa5941500f0 tx=0x7fa5940bdb30).stop 2023-10-18T17:58:16.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 shutdown_connections 2023-10-18T17:58:16.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa58005a540 0x7fa58005ca00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:16.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5940c27c0 0x7fa594146180 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:16.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa5940c23e0 0x7fa59414a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:16.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 --2- 172.21.15.72:0/2542488571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa594065670 0x7fa5940bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:16.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 >> 172.21.15.72:0/2542488571 conn(0x7fa5940620d0 msgr2=0x7fa5940624b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:16.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 shutdown_connections 2023-10-18T17:58:16.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.594+0000 7fa5992e4700 1 -- 172.21.15.72:0/2542488571 wait complete. 2023-10-18T17:58:16.600 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'rbd' created 2023-10-18T17:58:16.619 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-18T17:58:16.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.782+0000 7f8071c9b700 1 Processor -- start 2023-10-18T17:58:16.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.782+0000 7f8071c9b700 1 -- start start 2023-10-18T17:58:16.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f806c149d80 0x7f806c14a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:16.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c0689e0 0x7f806c0661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:16.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806c1500e0 con 0x7f806c0c69f0 2023-10-18T17:58:16.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f806c1503c0 con 0x7f806c149d80 2023-10-18T17:58:16.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c1506a0 con 0x7f806c0689e0 2023-10-18T17:58:16.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806bfff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f806c0c69f0 0x7f806c1517d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39138/0 (socket says 172.21.15.72:39138) 2023-10-18T17:58:16.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806bfff700 1 -- 172.21.15.72:0/1758305756 learned_addr learned my addr 172.21.15.72:0/1758305756 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:16.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806affd700 1 --2- 172.21.15.72:0/1758305756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c0689e0 0x7f806c0661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:16.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806b7fe700 1 --2- 172.21.15.72:0/1758305756 >> v2:172.21.15.179:3300/0 conn(0x7f806c149d80 0x7f806c14a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:16.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806a7fc700 1 -- 172.21.15.72:0/1758305756 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869475192 0 0) 0x7f806c1500e0 con 0x7f806c0c69f0 2023-10-18T17:58:16.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806a7fc700 1 -- 172.21.15.72:0/1758305756 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80540033f0 con 0x7f806c0c69f0 2023-10-18T17:58:16.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806b7fe700 1 -- 172.21.15.72:0/1758305756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c0689e0 msgr2=0x7f806c0661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:16.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806b7fe700 1 --2- 172.21.15.72:0/1758305756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c0689e0 0x7f806c0661c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:16.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806b7fe700 1 -- 172.21.15.72:0/1758305756 >> v1:172.21.15.72:6789/0 conn(0x7f806c0c69f0 legacy=0x7f806c1517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:16.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806b7fe700 1 -- 172.21.15.72:0/1758305756 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c150980 con 0x7f806c149d80 2023-10-18T17:58:16.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806affd700 1 --2- 172.21.15.72:0/1758305756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c0689e0 0x7f806c0661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:16.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806b7fe700 1 --2- 172.21.15.72:0/1758305756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c149d80 0x7f806c14a160 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f805c009760 tx=0x7f805c009060).ready entity=mon.1 client_cookie=b770db44c6d67a4d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:16.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806a7fc700 1 -- 172.21.15.72:0/1758305756 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00d4a0 con 0x7f806c149d80 2023-10-18T17:58:16.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806a7fc700 1 -- 172.21.15.72:0/1758305756 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c00d600 con 0x7f806c149d80 2023-10-18T17:58:16.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f806a7fc700 1 -- 172.21.15.72:0/1758305756 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00d980 con 0x7f806c149d80 2023-10-18T17:58:16.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 -- 172.21.15.72:0/1758305756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c149d80 msgr2=0x7f806c14a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:16.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 --2- 172.21.15.72:0/1758305756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c149d80 0x7f806c14a160 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f805c009760 tx=0x7f805c009060).stop 2023-10-18T17:58:16.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 -- 172.21.15.72:0/1758305756 shutdown_connections 2023-10-18T17:58:16.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 --2- 172.21.15.72:0/1758305756 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c0689e0 0x7f806c0661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:16.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 --2- 172.21.15.72:0/1758305756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c149d80 0x7f806c14a160 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:16.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 -- 172.21.15.72:0/1758305756 >> 172.21.15.72:0/1758305756 conn(0x7f806c05dc20 msgr2=0x7f806c05e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:16.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 -- 172.21.15.72:0/1758305756 shutdown_connections 2023-10-18T17:58:16.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.786+0000 7f8071c9b700 1 -- 172.21.15.72:0/1758305756 wait complete. 2023-10-18T17:58:16.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f8071c9b700 1 Processor -- start 2023-10-18T17:58:16.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f8071c9b700 1 -- start start 2023-10-18T17:58:16.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f8071c9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f806c0689e0 0x7f806c082fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:16.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f8071c9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c0c69f0 0x7f806c14be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:16.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f8071c9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c149d80 0x7f806c14d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:16.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f8071c9b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c1654d0 con 0x7f806c0689e0 2023-10-18T17:58:16.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f8071c9b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c14b180 con 0x7f806c0c69f0 2023-10-18T17:58:16.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f8071c9b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c14b460 con 0x7f806c149d80 2023-10-18T17:58:16.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f806b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f806c0689e0 0x7f806c082fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:16.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f806b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f806c0689e0 0x7f806c082fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49290/0 (socket says 172.21.15.72:49290) 2023-10-18T17:58:16.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f806b7fe700 1 -- 172.21.15.72:0/3701635303 learned_addr learned my addr 172.21.15.72:0/3701635303 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:16.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f806bfff700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c149d80 0x7f806c14d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:16.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.790+0000 7f806affd700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c0c69f0 0x7f806c14be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:16.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f806b7fe700 1 -- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c149d80 msgr2=0x7f806c14d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:16.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f806b7fe700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c149d80 0x7f806c14d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:16.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f806b7fe700 1 -- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c0c69f0 msgr2=0x7f806c14be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:16.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f806b7fe700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c0c69f0 0x7f806c14be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:16.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f806b7fe700 1 -- 172.21.15.72:0/3701635303 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80600092a0 con 0x7f806c0689e0 2023-10-18T17:58:16.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f806b7fe700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f806c0689e0 0x7f806c082fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f805c009d50 tx=0x7f805c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:16.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f80537fe700 1 -- 172.21.15.72:0/3701635303 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00dd30 con 0x7f806c0689e0 2023-10-18T17:58:16.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f80537fe700 1 -- 172.21.15.72:0/3701635303 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c00de90 con 0x7f806c0689e0 2023-10-18T17:58:16.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f80537fe700 1 -- 172.21.15.72:0/3701635303 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00d980 con 0x7f806c0689e0 2023-10-18T17:58:16.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c0092a0 con 0x7f806c0689e0 2023-10-18T17:58:16.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806c165160 con 0x7f806c0689e0 2023-10-18T17:58:16.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.794+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806c06c190 con 0x7f806c0689e0 2023-10-18T17:58:16.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.798+0000 7f80537fe700 1 -- 172.21.15.72:0/3701635303 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f805c01d640 con 0x7f806c0689e0 2023-10-18T17:58:16.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.798+0000 7f80537fe700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f805805a510 0x7f805805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:16.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.798+0000 7f80537fe700 1 -- 172.21.15.72:0/3701635303 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5258+0+0 (secure 0 0 0) 0x7f805c075be0 con 0x7f806c0689e0 2023-10-18T17:58:16.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.798+0000 7f806affd700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f805805a510 0x7f805805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:16.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.798+0000 7f806affd700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f805805a510 0x7f805805c9d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f806c0c36c0 tx=0x7f8060009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:16.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.806+0000 7f80537fe700 1 -- 172.21.15.72:0/3701635303 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f805c049660 con 0x7f806c0689e0 2023-10-18T17:58:16.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:16.974+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f806c05e740 con 0x7f806c0689e0 2023-10-18T17:58:17.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.594+0000 7f80537fe700 1 -- 172.21.15.72:0/3701635303 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f805c04cc80 con 0x7f806c0689e0 2023-10-18T17:58:17.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f805805a510 msgr2=0x7f805805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:17.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f805805a510 0x7f805805c9d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f806c0c36c0 tx=0x7f8060009060).stop 2023-10-18T17:58:17.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f806c0689e0 msgr2=0x7f806c082fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:17.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f806c0689e0 0x7f806c082fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f805c009d50 tx=0x7f805c016000).stop 2023-10-18T17:58:17.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 shutdown_connections 2023-10-18T17:58:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f805805a510 0x7f805805c9d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f806c149d80 0x7f806c14d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f806c0c69f0 0x7f806c14be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 --2- 172.21.15.72:0/3701635303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f806c0689e0 0x7f806c082fb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:17.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 >> 172.21.15.72:0/3701635303 conn(0x7f806c05dc20 msgr2=0x7f806c0c5110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:17.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 shutdown_connections 2023-10-18T17:58:17.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.602+0000 7f8071c9b700 1 -- 172.21.15.72:0/3701635303 wait complete. 2023-10-18T17:58:17.607 INFO:teuthology.orchestra.run.smithi072.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-18T17:58:17.626 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-18T17:58:17.627 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T17:58:17.627 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T17:58:17.631 INFO:tasks.ceph.mds.a:Started 2023-10-18T17:58:17.632 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T17:58:17.632 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T17:58:17.636 INFO:tasks.ceph.mds.c:Started 2023-10-18T17:58:17.636 INFO:tasks.ceph.mds.e:Restarting daemon 2023-10-18T17:58:17.637 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-10-18T17:58:17.641 INFO:tasks.ceph.mds.e:Started 2023-10-18T17:58:17.642 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T17:58:17.642 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T17:58:17.646 INFO:tasks.ceph.mds.b:Started 2023-10-18T17:58:17.646 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T17:58:17.646 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T17:58:17.649 INFO:tasks.ceph.mds.d:Started 2023-10-18T17:58:17.649 INFO:tasks.ceph.mds.f:Restarting daemon 2023-10-18T17:58:17.649 DEBUG:teuthology.orchestra.run.smithi179:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-10-18T17:58:17.651 INFO:tasks.ceph.mds.f:Started 2023-10-18T17:58:17.652 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-18T17:58:17.652 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:17.715 INFO:tasks.ceph.mds.d.smithi179.stdout:starting mds.d at 2023-10-18T17:58:17.717 INFO:tasks.ceph.mds.f.smithi179.stdout:starting mds.f at 2023-10-18T17:58:17.721 INFO:tasks.ceph.mds.b.smithi179.stdout:starting mds.b at 2023-10-18T17:58:17.728 INFO:tasks.ceph.mds.a.smithi072.stdout:starting mds.a at 2023-10-18T17:58:17.740 INFO:tasks.ceph.mds.e.smithi072.stdout:starting mds.e at 2023-10-18T17:58:17.741 INFO:tasks.ceph.mds.c.smithi072.stdout:starting mds.c at 2023-10-18T17:58:17.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53f1569700 1 Processor -- start 2023-10-18T17:58:17.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53f1569700 1 -- start start 2023-10-18T17:58:17.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53f1569700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f53ec0b9b20 0x7f53ec065b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:17.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53f1569700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f53ec153290 0x7f53ec15a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:17.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53f1569700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ec0681f0 con 0x7f53ec067d90 2023-10-18T17:58:17.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53f1569700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec0c0cf0 con 0x7f53ec153290 2023-10-18T17:58:17.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53f1569700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec0c2b70 con 0x7f53ec0b9b20 2023-10-18T17:58:17.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53eb7fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f53ec153290 0x7f53ec15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:17.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53eb7fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f53ec153290 0x7f53ec15a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49322/0 (socket says 172.21.15.72:49322) 2023-10-18T17:58:17.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.798+0000 7f53eb7fe700 1 -- 172.21.15.72:0/3581268948 learned_addr learned my addr 172.21.15.72:0/3581268948 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:17.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53eaffd700 1 --2- 172.21.15.72:0/3581268948 >> v2:172.21.15.179:3300/0 conn(0x7f53ec0b9b20 0x7f53ec065b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:17.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53eb7fe700 1 -- 172.21.15.72:0/3581268948 >> v2:172.21.15.179:3300/0 conn(0x7f53ec0b9b20 msgr2=0x7f53ec065b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:17.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53eb7fe700 1 --2- 172.21.15.72:0/3581268948 >> v2:172.21.15.179:3300/0 conn(0x7f53ec0b9b20 0x7f53ec065b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:17.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53eb7fe700 1 -- 172.21.15.72:0/3581268948 >> v1:172.21.15.179:6789/0 conn(0x7f53ec067d90 legacy=0x7f53ec0645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:17.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53eb7fe700 1 -- 172.21.15.72:0/3581268948 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec1536e0 con 0x7f53ec153290 2023-10-18T17:58:17.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53eb7fe700 1 --2- 172.21.15.72:0/3581268948 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec153290 0x7f53ec15a630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f53ec0c73c0 tx=0x7f53e4009840).ready entity=mon.0 client_cookie=fc99f1b5cb33d5da server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:17.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53e9ffb700 1 -- 172.21.15.72:0/3581268948 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400e070 con 0x7f53ec153290 2023-10-18T17:58:17.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53e9ffb700 1 -- 172.21.15.72:0/3581268948 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e400ba20 con 0x7f53ec153290 2023-10-18T17:58:17.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53e9ffb700 1 -- 172.21.15.72:0/3581268948 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400bda0 con 0x7f53ec153290 2023-10-18T17:58:17.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53f1569700 1 -- 172.21.15.72:0/3581268948 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec153290 msgr2=0x7f53ec15a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:17.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53f1569700 1 --2- 172.21.15.72:0/3581268948 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec153290 0x7f53ec15a630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f53ec0c73c0 tx=0x7f53e4009840).stop 2023-10-18T17:58:17.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53f1569700 1 -- 172.21.15.72:0/3581268948 shutdown_connections 2023-10-18T17:58:17.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53f1569700 1 --2- 172.21.15.72:0/3581268948 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec153290 0x7f53ec15a630 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:17.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53f1569700 1 --2- 172.21.15.72:0/3581268948 >> v2:172.21.15.179:3300/0 conn(0x7f53ec0b9b20 0x7f53ec065b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:17.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53f1569700 1 -- 172.21.15.72:0/3581268948 >> 172.21.15.72:0/3581268948 conn(0x7f53ec069c80 msgr2=0x7f53ec06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:17.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53f1569700 1 -- 172.21.15.72:0/3581268948 shutdown_connections 2023-10-18T17:58:17.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.802+0000 7f53f1569700 1 -- 172.21.15.72:0/3581268948 wait complete. 2023-10-18T17:58:17.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.806+0000 7f53f1569700 1 Processor -- start 2023-10-18T17:58:17.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.806+0000 7f53f1569700 1 -- start start 2023-10-18T17:58:17.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53f1569700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec067d90 0x7f53ec150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:17.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53f1569700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53ec0b9b20 0x7f53ec1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:17.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53f1569700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53ec153290 0x7f53ec152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:17.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53f1569700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec146c70 con 0x7f53ec067d90 2023-10-18T17:58:17.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53f1569700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0bf7c0 con 0x7f53ec0b9b20 2023-10-18T17:58:17.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53f1569700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0bfbf0 con 0x7f53ec153290 2023-10-18T17:58:17.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec067d90 0x7f53ec150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:17.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec067d90 0x7f53ec150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49328/0 (socket says 172.21.15.72:49328) 2023-10-18T17:58:17.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eaffd700 1 -- 172.21.15.72:0/1593718895 learned_addr learned my addr 172.21.15.72:0/1593718895 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:17.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eb7fe700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53ec153290 0x7f53ec152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:17.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53ea7fc700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53ec0b9b20 0x7f53ec1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:17.816 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eaffd700 1 -- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53ec153290 msgr2=0x7f53ec152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:17.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eaffd700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53ec153290 0x7f53ec152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:17.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eaffd700 1 -- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53ec0b9b20 msgr2=0x7f53ec1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:17.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eaffd700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53ec0b9b20 0x7f53ec1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:17.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eaffd700 1 -- 172.21.15.72:0/1593718895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e4009ae0 con 0x7f53ec067d90 2023-10-18T17:58:17.817 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53eaffd700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec067d90 0x7f53ec150880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f53dc00e960 tx=0x7f53dc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:17.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53d2ffd700 1 -- 172.21.15.72:0/1593718895 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc012600 con 0x7f53ec067d90 2023-10-18T17:58:17.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53d2ffd700 1 -- 172.21.15.72:0/1593718895 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc012760 con 0x7f53ec067d90 2023-10-18T17:58:17.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53d2ffd700 1 -- 172.21.15.72:0/1593718895 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc012ae0 con 0x7f53ec067d90 2023-10-18T17:58:17.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec146990 con 0x7f53ec067d90 2023-10-18T17:58:17.818 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.810+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec144da0 con 0x7f53ec067d90 2023-10-18T17:58:17.819 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.814+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec0bfdf0 con 0x7f53ec067d90 2023-10-18T17:58:17.819 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.814+0000 7f53d2ffd700 1 -- 172.21.15.72:0/1593718895 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f53dc01b400 con 0x7f53ec067d90 2023-10-18T17:58:17.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.814+0000 7f53d2ffd700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f53d805a4e0 0x7f53d805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:17.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.814+0000 7f53d2ffd700 1 -- 172.21.15.72:0/1593718895 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f53dc074980 con 0x7f53ec067d90 2023-10-18T17:58:17.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.814+0000 7f53ea7fc700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f53d805a4e0 0x7f53d805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:17.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.818+0000 7f53ea7fc700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f53d805a4e0 0x7f53d805c9a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f53ec0c3d70 tx=0x7f53e0007d40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:17.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.822+0000 7f53d2ffd700 1 -- 172.21.15.72:0/1593718895 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f53ec0bfdf0 con 0x7f53ec067d90 2023-10-18T17:58:17.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.994+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53ec0bf2b0 con 0x7f53ec067d90 2023-10-18T17:58:17.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.994+0000 7f53d2ffd700 1 -- 172.21.15.72:0/1593718895 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13722 (secure 0 0 0) 0x7f53ec0bf2b0 con 0x7f53ec067d90 2023-10-18T17:58:17.999 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:17.999 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":17,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:17.589826+0000","last_up_change":"2023-10-18T17:58:14.550597+0000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T17:58:13.627096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T17:58:15.732786+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":33232},{"type":"v1","addr":"172.21.15.72:6810","nonce":33232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6813","nonce":33232},{"type":"v1","addr":"172.21.15.72:6815","nonce":33232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":33232},{"type":"v1","addr":"172.21.15.72:6826","nonce":33232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":33232},{"type":"v1","addr":"172.21.15.72:6820","nonce":33232}]},"public_addr":"172.21.15.72:6810/33232","cluster_addr":"172.21.15.72:6815/33232","heartbeat_back_addr":"172.21.15.72:6826/33232","heartbeat_front_addr":"172.21.15.72:6820/33232","state":["exists","up"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6809","nonce":33229},{"type":"v1","addr":"172.21.15.72:6812","nonce":33229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":33229},{"type":"v1","addr":"172.21.15.72:6819","nonce":33229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":33229},{"type":"v1","addr":"172.21.15.72:6831","nonce":33229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":33229},{"type":"v1","addr":"172.21.15.72:6825","nonce":33229}]},"public_addr":"172.21.15.72:6812/33229","cluster_addr":"172.21.15.72:6819/33229","heartbeat_back_addr":"172.21.15.72:6831/33229","heartbeat_front_addr":"172.21.15.72:6825/33229","state":["exists","up"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":33231},{"type":"v1","addr":"172.21.15.72:6801","nonce":33231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":33231},{"type":"v1","addr":"172.21.15.72:6803","nonce":33231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":33231},{"type":"v1","addr":"172.21.15.72:6807","nonce":33231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":33231},{"type":"v1","addr":"172.21.15.72:6805","nonce":33231}]},"public_addr":"172.21.15.72:6801/33231","cluster_addr":"172.21.15.72:6803/33231","heartbeat_back_addr":"172.21.15.72:6807/33231","heartbeat_front_addr":"172.21.15.72:6805/33231","state":["exists","up"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":33228},{"type":"v1","addr":"172.21.15.72:6814","nonce":33228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":33228},{"type":"v1","addr":"172.21.15.72:6821","nonce":33228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":33228},{"type":"v1","addr":"172.21.15.72:6830","nonce":33228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":33228},{"type":"v1","addr":"172.21.15.72:6827","nonce":33228}]},"public_addr":"172.21.15.72:6814/33228","cluster_addr":"172.21.15.72:6821/33228","heartbeat_back_addr":"172.21.15.72:6830/33228","heartbeat_front_addr":"172.21.15.72:6827/33228","state":["exists","up"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":34034},{"type":"v1","addr":"172.21.15.179:6809","nonce":34034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":34034},{"type":"v1","addr":"172.21.15.179:6813","nonce":34034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":34034},{"type":"v1","addr":"172.21.15.179:6821","nonce":34034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":34034},{"type":"v1","addr":"172.21.15.179:6817","nonce":34034}]},"public_addr":"172.21.15.179:6809/34034","cluster_addr":"172.21.15.179:6813/34034","heartbeat_back_addr":"172.21.15.179:6821/34034","heartbeat_front_addr":"172.21.15.179:6817/34034","state":["exists","up"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":34044},{"type":"v1","addr":"172.21.15.179:6801","nonce":34044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":34044},{"type":"v1","addr":"172.21.15.179:6803","nonce":34044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":34044},{"type":"v1","addr":"172.21.15.179:6807","nonce":34044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":34044},{"type":"v1","addr":"172.21.15.179:6805","nonce":34044}]},"public_addr":"172.21.15.179:6801/34044","cluster_addr":"172.21.15.179:6803/34044","heartbeat_back_addr":"172.21.15.179:6807/34044","heartbeat_front_addr":"172.21.15.179:6805/34044","state":["exists","up"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":34036},{"type":"v1","addr":"172.21.15.179:6825","nonce":34036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":34036},{"type":"v1","addr":"172.21.15.179:6827","nonce":34036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":34036},{"type":"v1","addr":"172.21.15.179:6831","nonce":34036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":34036},{"type":"v1","addr":"172.21.15.179:6829","nonce":34036}]},"public_addr":"172.21.15.179:6825/34036","cluster_addr":"172.21.15.179:6827/34036","heartbeat_back_addr":"172.21.15.179:6831/34036","heartbeat_front_addr":"172.21.15.179:6829/34036","state":["exists","up"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":34035},{"type":"v1","addr":"172.21.15.179:6812","nonce":34035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":34035},{"type":"v1","addr":"172.21.15.179:6816","nonce":34035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":34035},{"type":"v1","addr":"172.21.15.179:6823","nonce":34035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":34035},{"type":"v1","addr":"172.21.15.179:6820","nonce":34035}]},"public_addr":"172.21.15.179:6812/34035","cluster_addr":"172.21.15.179:6816/34035","heartbeat_back_addr":"172.21.15.179:6823/34035","heartbeat_front_addr":"172.21.15.179:6820/34035","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.697876+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.661119+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.579136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.792036+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.069244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:11.971339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.176699+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.197004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:18.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f53d805a4e0 msgr2=0x7f53d805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f53d805a4e0 0x7f53d805c9a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f53ec0c3d70 tx=0x7f53e0007d40).stop 2023-10-18T17:58:18.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec067d90 msgr2=0x7f53ec150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec067d90 0x7f53ec150880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f53dc00e960 tx=0x7f53dc009060).stop 2023-10-18T17:58:18.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 shutdown_connections 2023-10-18T17:58:18.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f53d805a4e0 0x7f53d805c9a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f53ec153290 0x7f53ec152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53ec0b9b20 0x7f53ec1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 --2- 172.21.15.72:0/1593718895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53ec067d90 0x7f53ec150880 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 >> 172.21.15.72:0/1593718895 conn(0x7f53ec069c80 msgr2=0x7f53ec0c5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:18.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 shutdown_connections 2023-10-18T17:58:18.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:17.998+0000 7f53f1569700 1 -- 172.21.15.72:0/1593718895 wait complete. 2023-10-18T17:58:18.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-10-18T17:58:13.627096+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T17:58:15.732786+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-18T17:58:18.023 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-10-18T17:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.198+0000 7f5ed5379700 1 Processor -- start 2023-10-18T17:58:18.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.198+0000 7f5ed5379700 1 -- start start 2023-10-18T17:58:18.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.202+0000 7f5ed5379700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5ed00c11f0 0x7f5ed00c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.202+0000 7f5ed5379700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5ed0065670 0x7f5ed0065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.202+0000 7f5ed5379700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed014ffb0 con 0x7f5ed00c23e0 2023-10-18T17:58:18.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.202+0000 7f5ed5379700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed0150df0 con 0x7f5ed0065670 2023-10-18T17:58:18.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.202+0000 7f5ed5379700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed0150270 con 0x7f5ed00c11f0 2023-10-18T17:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.202+0000 7f5ed4b78700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5ed0065670 0x7f5ed0065ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.202+0000 7f5ed4b78700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5ed0065670 0x7f5ed0065ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49344/0 (socket says 172.21.15.72:49344) 2023-10-18T17:58:18.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.202+0000 7f5ed4b78700 1 -- 172.21.15.72:0/2631647394 learned_addr learned my addr 172.21.15.72:0/2631647394 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:18.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.202+0000 7f5ecf7fe700 1 --2- 172.21.15.72:0/2631647394 >> v2:172.21.15.179:3300/0 conn(0x7f5ed00c11f0 0x7f5ed00c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed4b78700 1 -- 172.21.15.72:0/2631647394 >> v2:172.21.15.179:3300/0 conn(0x7f5ed00c11f0 msgr2=0x7f5ed00c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed4b78700 1 --2- 172.21.15.72:0/2631647394 >> v2:172.21.15.179:3300/0 conn(0x7f5ed00c11f0 0x7f5ed00c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed4b78700 1 -- 172.21.15.72:0/2631647394 >> v1:172.21.15.72:6789/0 conn(0x7f5ed00c23e0 legacy=0x7f5ed00664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed4b78700 1 -- 172.21.15.72:0/2631647394 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed0150550 con 0x7f5ed0065670 2023-10-18T17:58:18.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5eceffd700 1 -- 172.21.15.72:0/2631647394 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277249587 0 0) 0x7f5ed014ffb0 con 0x7f5ed00c23e0 2023-10-18T17:58:18.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed4b78700 1 --2- 172.21.15.72:0/2631647394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 0x7f5ed0065ac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5ed0064bd0 tx=0x7f5ec8009840).ready entity=mon.0 client_cookie=c24cdc5bfefe1e63 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:18.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5eceffd700 1 -- 172.21.15.72:0/2631647394 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800e070 con 0x7f5ed0065670 2023-10-18T17:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5eceffd700 1 -- 172.21.15.72:0/2631647394 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec800ba20 con 0x7f5ed0065670 2023-10-18T17:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5eceffd700 1 -- 172.21.15.72:0/2631647394 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800bda0 con 0x7f5ed0065670 2023-10-18T17:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed5379700 1 -- 172.21.15.72:0/2631647394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 msgr2=0x7f5ed0065ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed5379700 1 --2- 172.21.15.72:0/2631647394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 0x7f5ed0065ac0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f5ed0064bd0 tx=0x7f5ec8009840).stop 2023-10-18T17:58:18.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed5379700 1 -- 172.21.15.72:0/2631647394 shutdown_connections 2023-10-18T17:58:18.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed5379700 1 --2- 172.21.15.72:0/2631647394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 0x7f5ed0065ac0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed5379700 1 --2- 172.21.15.72:0/2631647394 >> v2:172.21.15.179:3300/0 conn(0x7f5ed00c11f0 0x7f5ed00c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed5379700 1 -- 172.21.15.72:0/2631647394 >> 172.21.15.72:0/2631647394 conn(0x7f5ed00620d0 msgr2=0x7f5ed00624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:18.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed5379700 1 -- 172.21.15.72:0/2631647394 shutdown_connections 2023-10-18T17:58:18.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.206+0000 7f5ed5379700 1 -- 172.21.15.72:0/2631647394 wait complete. 2023-10-18T17:58:18.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ed5379700 1 Processor -- start 2023-10-18T17:58:18.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ed5379700 1 -- start start 2023-10-18T17:58:18.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ed5379700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 0x7f5ed0155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ed5379700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ed00c11f0 0x7f5ed0163600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ed5379700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed00c23e0 0x7f5ed014bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ed5379700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed01439b0 con 0x7f5ed0065670 2023-10-18T17:58:18.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ed5379700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed01433d0 con 0x7f5ed00c23e0 2023-10-18T17:58:18.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ed5379700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed014a680 con 0x7f5ed00c11f0 2023-10-18T17:58:18.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ecffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 0x7f5ed0155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ecffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 0x7f5ed0155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49348/0 (socket says 172.21.15.72:49348) 2023-10-18T17:58:18.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ecffff700 1 -- 172.21.15.72:0/1579326070 learned_addr learned my addr 172.21.15.72:0/1579326070 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:18.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ecf7fe700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ed00c11f0 0x7f5ed0163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ed4b78700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed00c23e0 0x7f5ed014bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ecffff700 1 -- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ed00c11f0 msgr2=0x7f5ed0163600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ecffff700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ed00c11f0 0x7f5ed0163600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ecffff700 1 -- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed00c23e0 msgr2=0x7f5ed014bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ecffff700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed00c23e0 0x7f5ed014bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.210+0000 7f5ecffff700 1 -- 172.21.15.72:0/1579326070 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec8009ae0 con 0x7f5ed0065670 2023-10-18T17:58:18.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.214+0000 7f5ecffff700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 0x7f5ed0155dc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5ec000c470 tx=0x7f5ec0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:18.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.214+0000 7f5eb7fff700 1 -- 172.21.15.72:0/1579326070 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0007020 con 0x7f5ed0065670 2023-10-18T17:58:18.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.214+0000 7f5eb7fff700 1 -- 172.21.15.72:0/1579326070 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec00071a0 con 0x7f5ed0065670 2023-10-18T17:58:18.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.214+0000 7f5eb7fff700 1 -- 172.21.15.72:0/1579326070 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0007560 con 0x7f5ed0065670 2023-10-18T17:58:18.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.214+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed014ae70 con 0x7f5ed0065670 2023-10-18T17:58:18.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.214+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed0143640 con 0x7f5ed0065670 2023-10-18T17:58:18.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.214+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed0046f00 con 0x7f5ed0065670 2023-10-18T17:58:18.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.214+0000 7f5eb7fff700 1 -- 172.21.15.72:0/1579326070 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5ec00068b0 con 0x7f5ed0065670 2023-10-18T17:58:18.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.218+0000 7f5eb7fff700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5ebc05a540 0x7f5ebc05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.218+0000 7f5eb7fff700 1 -- 172.21.15.72:0/1579326070 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f5ec00737e0 con 0x7f5ed0065670 2023-10-18T17:58:18.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.218+0000 7f5ecf7fe700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5ebc05a540 0x7f5ebc05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.218+0000 7f5ecf7fe700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5ebc05a540 0x7f5ebc05ca00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5ed00c0e90 tx=0x7f5ec4009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:18.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.222+0000 7f5eb7fff700 1 -- 172.21.15.72:0/1579326070 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f5ec0047250 con 0x7f5ed0065670 2023-10-18T17:58:18.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.386+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5ed0150830 con 0x7f5ed0065670 2023-10-18T17:58:18.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.386+0000 7f5eb7fff700 1 -- 172.21.15.72:0/1579326070 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5ec004a870 con 0x7f5ed0065670 2023-10-18T17:58:18.390 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-10-18T17:58:18.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5ebc05a540 msgr2=0x7f5ebc05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5ebc05a540 0x7f5ebc05ca00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5ed00c0e90 tx=0x7f5ec4009060).stop 2023-10-18T17:58:18.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 msgr2=0x7f5ed0155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 0x7f5ed0155dc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5ec000c470 tx=0x7f5ec0007e00).stop 2023-10-18T17:58:18.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 shutdown_connections 2023-10-18T17:58:18.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5ebc05a540 0x7f5ebc05ca00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed00c23e0 0x7f5ed014bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5ed00c11f0 0x7f5ed0163600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 --2- 172.21.15.72:0/1579326070 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ed0065670 0x7f5ed0155dc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 >> 172.21.15.72:0/1579326070 conn(0x7f5ed00620d0 msgr2=0x7f5ed0065a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:18.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 shutdown_connections 2023-10-18T17:58:18.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.390+0000 7f5ed5379700 1 -- 172.21.15.72:0/1579326070 wait complete. 2023-10-18T17:58:18.414 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T17:58:18.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.570+0000 7f95b7279700 1 Processor -- start 2023-10-18T17:58:18.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.570+0000 7f95b7279700 1 -- start start 2023-10-18T17:58:18.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b7279700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f95b00c23e0 0x7f95b00c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b7279700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b014ffb0 con 0x7f95b0065670 2023-10-18T17:58:18.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b7279700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b0150270 con 0x7f95b00c27c0 2023-10-18T17:58:18.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b7279700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b0150df0 con 0x7f95b00c23e0 2023-10-18T17:58:18.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b6277700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f95b00c23e0 0x7f95b00c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b6277700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f95b00c23e0 0x7f95b00c11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:34870/0 (socket says 172.21.15.72:34870) 2023-10-18T17:58:18.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b6277700 1 -- 172.21.15.72:0/4087478659 learned_addr learned my addr 172.21.15.72:0/4087478659 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:18.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b5275700 1 -- 172.21.15.72:0/4087478659 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856021972 0 0) 0x7f95b0150270 con 0x7f95b00c27c0 2023-10-18T17:58:18.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b6277700 1 -- 172.21.15.72:0/4087478659 >> v1:172.21.15.179:6789/0 conn(0x7f95b00c27c0 legacy=0x7f95b00664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b6277700 1 -- 172.21.15.72:0/4087478659 >> v1:172.21.15.72:6789/0 conn(0x7f95b0065670 legacy=0x7f95b0065a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b6277700 1 -- 172.21.15.72:0/4087478659 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b0150550 con 0x7f95b00c23e0 2023-10-18T17:58:18.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.574+0000 7f95b6277700 1 --2- 172.21.15.72:0/4087478659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b00c23e0 0x7f95b00c11f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f95ac006fd0 tx=0x7f95ac0090f0).ready entity=mon.1 client_cookie=bb900f306f6d031a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:18.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b5275700 1 -- 172.21.15.72:0/4087478659 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00f510 con 0x7f95b00c23e0 2023-10-18T17:58:18.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b5275700 1 -- 172.21.15.72:0/4087478659 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac00f670 con 0x7f95b00c23e0 2023-10-18T17:58:18.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b5275700 1 -- 172.21.15.72:0/4087478659 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00fa10 con 0x7f95b00c23e0 2023-10-18T17:58:18.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b7279700 1 -- 172.21.15.72:0/4087478659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b00c23e0 msgr2=0x7f95b00c11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b7279700 1 --2- 172.21.15.72:0/4087478659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b00c23e0 0x7f95b00c11f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f95ac006fd0 tx=0x7f95ac0090f0).stop 2023-10-18T17:58:18.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b7279700 1 -- 172.21.15.72:0/4087478659 shutdown_connections 2023-10-18T17:58:18.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b7279700 1 --2- 172.21.15.72:0/4087478659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b00c23e0 0x7f95b00c11f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b7279700 1 -- 172.21.15.72:0/4087478659 >> 172.21.15.72:0/4087478659 conn(0x7f95b00620d0 msgr2=0x7f95b00624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:18.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b7279700 1 -- 172.21.15.72:0/4087478659 shutdown_connections 2023-10-18T17:58:18.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.578+0000 7f95b7279700 1 -- 172.21.15.72:0/4087478659 wait complete. 2023-10-18T17:58:18.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b7279700 1 Processor -- start 2023-10-18T17:58:18.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b7279700 1 -- start start 2023-10-18T17:58:18.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b7279700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b0065670 0x7f95b0163570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b7279700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95b00c23e0 0x7f95b0151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b7279700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b00c27c0 0x7f95b0151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b7279700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00c5d60 con 0x7f95b00c27c0 2023-10-18T17:58:18.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b7279700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0152aa0 con 0x7f95b0065670 2023-10-18T17:58:18.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b7279700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0151610 con 0x7f95b00c23e0 2023-10-18T17:58:18.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6a78700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b00c27c0 0x7f95b0151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6277700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b0065670 0x7f95b0163570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6a78700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b00c27c0 0x7f95b0151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49356/0 (socket says 172.21.15.72:49356) 2023-10-18T17:58:18.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6a78700 1 -- 172.21.15.72:0/2167524398 learned_addr learned my addr 172.21.15.72:0/2167524398 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:18.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6277700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b0065670 0x7f95b0163570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:34878/0 (socket says 172.21.15.72:34878) 2023-10-18T17:58:18.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b5a76700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95b00c23e0 0x7f95b0151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6a78700 1 -- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95b00c23e0 msgr2=0x7f95b0151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6a78700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95b00c23e0 0x7f95b0151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6a78700 1 -- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b0065670 msgr2=0x7f95b0163570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6a78700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b0065670 0x7f95b0163570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6a78700 1 -- 172.21.15.72:0/2167524398 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac009370 con 0x7f95b00c27c0 2023-10-18T17:58:18.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95b6a78700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b00c27c0 0x7f95b0151f90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f95a800c890 tx=0x7f95a8001610).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:18.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95a67fc700 1 -- 172.21.15.72:0/2167524398 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800a530 con 0x7f95b00c27c0 2023-10-18T17:58:18.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95a67fc700 1 -- 172.21.15.72:0/2167524398 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a800a6b0 con 0x7f95b00c27c0 2023-10-18T17:58:18.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.582+0000 7f95a67fc700 1 -- 172.21.15.72:0/2167524398 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800aa70 con 0x7f95b00c27c0 2023-10-18T17:58:18.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.586+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b014edb0 con 0x7f95b00c27c0 2023-10-18T17:58:18.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.586+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b0163fa0 con 0x7f95b00c27c0 2023-10-18T17:58:18.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.586+0000 7f95a67fc700 1 -- 172.21.15.72:0/2167524398 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f95a801b400 con 0x7f95b00c27c0 2023-10-18T17:58:18.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.586+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b0046f00 con 0x7f95b00c27c0 2023-10-18T17:58:18.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.586+0000 7f95a67fc700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f959405a540 0x7f959405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.586+0000 7f95a67fc700 1 -- 172.21.15.72:0/2167524398 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f95a80743b0 con 0x7f95b00c27c0 2023-10-18T17:58:18.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.586+0000 7f95b6277700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f959405a540 0x7f959405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.590+0000 7f95b6277700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f959405a540 0x7f959405ca00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f95ac006cc0 tx=0x7f95ac017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:18.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.594+0000 7f95a67fc700 1 -- 172.21.15.72:0/2167524398 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f95a8047e80 con 0x7f95b00c27c0 2023-10-18T17:58:18.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.770+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f95b0150830 con 0x7f95b00c27c0 2023-10-18T17:58:18.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.770+0000 7f95a67fc700 1 -- 172.21.15.72:0/2167524398 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f95a804b4a0 con 0x7f95b00c27c0 2023-10-18T17:58:18.775 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 8 2023-10-18T17:58:18.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f959405a540 msgr2=0x7f959405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f959405a540 0x7f959405ca00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f95ac006cc0 tx=0x7f95ac017000).stop 2023-10-18T17:58:18.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b00c27c0 msgr2=0x7f95b0151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b00c27c0 0x7f95b0151f90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f95a800c890 tx=0x7f95a8001610).stop 2023-10-18T17:58:18.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 shutdown_connections 2023-10-18T17:58:18.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f959405a540 0x7f959405ca00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95b00c27c0 0x7f95b0151f90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95b00c23e0 0x7f95b0151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 --2- 172.21.15.72:0/2167524398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95b0065670 0x7f95b0163570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 >> 172.21.15.72:0/2167524398 conn(0x7f95b00620d0 msgr2=0x7f95b0066590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:18.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 shutdown_connections 2023-10-18T17:58:18.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.774+0000 7f95b7279700 1 -- 172.21.15.72:0/2167524398 wait complete. 2023-10-18T17:58:18.799 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T17:58:18.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.962+0000 7f29e5546700 1 Processor -- start 2023-10-18T17:58:18.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.962+0000 7f29e5546700 1 -- start start 2023-10-18T17:58:18.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29e5546700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f29e0149660 0x7f29e00be490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29e5546700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e00c0fb0 0x7f29e00be980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29e5546700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e00698e0 con 0x7f29e0149280 2023-10-18T17:58:18.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29e5546700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e00c3d70 con 0x7f29e0149660 2023-10-18T17:58:18.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29e5546700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00c4010 con 0x7f29e00c0fb0 2023-10-18T17:58:18.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29de7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f29e0149660 0x7f29e00be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29de7fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f29e0149660 0x7f29e00be490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:34884/0 (socket says 172.21.15.72:34884) 2023-10-18T17:58:18.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29de7fc700 1 -- 172.21.15.72:0/3355730938 learned_addr learned my addr 172.21.15.72:0/3355730938 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:18.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29dffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e00c0fb0 0x7f29e00be980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29deffd700 1 --1- 172.21.15.72:0/3355730938 >> v1:172.21.15.179:6789/0 conn(0x7f29e0149280 0x7f29e00c0520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:46822/0 (socket says 172.21.15.72:46822) 2023-10-18T17:58:18.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29de7fc700 1 -- 172.21.15.72:0/3355730938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e00c0fb0 msgr2=0x7f29e00be980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:18.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29de7fc700 1 --2- 172.21.15.72:0/3355730938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e00c0fb0 0x7f29e00be980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29de7fc700 1 -- 172.21.15.72:0/3355730938 >> v1:172.21.15.179:6789/0 conn(0x7f29e0149280 legacy=0x7f29e00c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29de7fc700 1 -- 172.21.15.72:0/3355730938 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e00c42b0 con 0x7f29e0149660 2023-10-18T17:58:18.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.966+0000 7f29de7fc700 1 --2- 172.21.15.72:0/3355730938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e0149660 0x7f29e00be490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f29d4006fd0 tx=0x7f29d40090f0).ready entity=mon.1 client_cookie=347cb45afc5979b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:18.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29ddffb700 1 -- 172.21.15.72:0/3355730938 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d400b610 con 0x7f29e0149660 2023-10-18T17:58:18.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29ddffb700 1 -- 172.21.15.72:0/3355730938 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d400b770 con 0x7f29e0149660 2023-10-18T17:58:18.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29ddffb700 1 -- 172.21.15.72:0/3355730938 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d400baf0 con 0x7f29e0149660 2023-10-18T17:58:18.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29e5546700 1 -- 172.21.15.72:0/3355730938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e0149660 msgr2=0x7f29e00be490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29e5546700 1 --2- 172.21.15.72:0/3355730938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e0149660 0x7f29e00be490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f29d4006fd0 tx=0x7f29d40090f0).stop 2023-10-18T17:58:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29e5546700 1 -- 172.21.15.72:0/3355730938 shutdown_connections 2023-10-18T17:58:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29e5546700 1 --2- 172.21.15.72:0/3355730938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e00c0fb0 0x7f29e00be980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29e5546700 1 --2- 172.21.15.72:0/3355730938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e0149660 0x7f29e00be490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29e5546700 1 -- 172.21.15.72:0/3355730938 >> 172.21.15.72:0/3355730938 conn(0x7f29e0067510 msgr2=0x7f29e0067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29e5546700 1 -- 172.21.15.72:0/3355730938 shutdown_connections 2023-10-18T17:58:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.970+0000 7f29e5546700 1 -- 172.21.15.72:0/3355730938 wait complete. 2023-10-18T17:58:18.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29e5546700 1 Processor -- start 2023-10-18T17:58:18.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29e5546700 1 -- start start 2023-10-18T17:58:18.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29e5546700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e00c0fb0 0x7f29e0147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29e5546700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e0149280 0x7f29e0144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29e5546700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e0149660 0x7f29e00b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29e5546700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e014f740 con 0x7f29e00c0fb0 2023-10-18T17:58:18.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29e5546700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00c2540 con 0x7f29e0149280 2023-10-18T17:58:18.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29e5546700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0148060 con 0x7f29e0149660 2023-10-18T17:58:18.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29deffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e00c0fb0 0x7f29e0147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29deffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e00c0fb0 0x7f29e0147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49362/0 (socket says 172.21.15.72:49362) 2023-10-18T17:58:18.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29deffd700 1 -- 172.21.15.72:0/2209452647 learned_addr learned my addr 172.21.15.72:0/2209452647 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:18.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29dffff700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e0149660 0x7f29e00b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.974+0000 7f29de7fc700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e0149280 0x7f29e0144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29deffd700 1 -- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e0149660 msgr2=0x7f29e00b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29deffd700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e0149660 0x7f29e00b8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29deffd700 1 -- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e0149280 msgr2=0x7f29e0144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:18.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29deffd700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e0149280 0x7f29e0144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:18.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29deffd700 1 -- 172.21.15.72:0/2209452647 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4009370 con 0x7f29e00c0fb0 2023-10-18T17:58:18.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29deffd700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e00c0fb0 0x7f29e0147a40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f29d000c490 tx=0x7f29d0007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:18.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29c6ffd700 1 -- 172.21.15.72:0/2209452647 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d000ce00 con 0x7f29e00c0fb0 2023-10-18T17:58:18.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29c6ffd700 1 -- 172.21.15.72:0/2209452647 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d00072e0 con 0x7f29e00c0fb0 2023-10-18T17:58:18.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e014f950 con 0x7f29e00c0fb0 2023-10-18T17:58:18.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29c6ffd700 1 -- 172.21.15.72:0/2209452647 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00056e0 con 0x7f29e00c0fb0 2023-10-18T17:58:18.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e014ff10 con 0x7f29e00c0fb0 2023-10-18T17:58:18.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.978+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e0046f00 con 0x7f29e00c0fb0 2023-10-18T17:58:18.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.982+0000 7f29c6ffd700 1 -- 172.21.15.72:0/2209452647 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f29d0005b10 con 0x7f29e00c0fb0 2023-10-18T17:58:18.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.982+0000 7f29c6ffd700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f29cc05a510 0x7f29cc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:18.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.982+0000 7f29c6ffd700 1 -- 172.21.15.72:0/2209452647 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f29d0073830 con 0x7f29e00c0fb0 2023-10-18T17:58:18.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.982+0000 7f29de7fc700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f29cc05a510 0x7f29cc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:18.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.986+0000 7f29de7fc700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f29cc05a510 0x7f29cc05c9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f29d4009130 tx=0x7f29d4017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:18.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:18.990+0000 7f29c6ffd700 1 -- 172.21.15.72:0/2209452647 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f29d00472a0 con 0x7f29e00c0fb0 2023-10-18T17:58:19.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.186+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29e00b88b0 con 0x7f29e00c0fb0 2023-10-18T17:58:19.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.186+0000 7f29c6ffd700 1 -- 172.21.15.72:0/2209452647 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5039 (secure 0 0 0) 0x7f29d0018070 con 0x7f29e00c0fb0 2023-10-18T17:58:19.192 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:19.192 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/3359824101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3359824101},{"type":"v1","addr":"172.21.15.72:6836","nonce":3359824101}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3056786712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3056786712},{"type":"v1","addr":"172.21.15.179:6835","nonce":3056786712}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/4028468466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":4028468466},{"type":"v1","addr":"172.21.15.179:6839","nonce":4028468466}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1416523495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":1416523495},{"type":"v1","addr":"172.21.15.72:6837","nonce":1416523495}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/814982392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":814982392},{"type":"v1","addr":"172.21.15.179:6837","nonce":814982392}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4294,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/247898707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":247898707},{"type":"v1","addr":"172.21.15.72:6833","nonce":247898707}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-18T17:58:19.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f29cc05a510 msgr2=0x7f29cc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f29cc05a510 0x7f29cc05c9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f29d4009130 tx=0x7f29d4017000).stop 2023-10-18T17:58:19.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e00c0fb0 msgr2=0x7f29e0147a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e00c0fb0 0x7f29e0147a40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f29d000c490 tx=0x7f29d0007df0).stop 2023-10-18T17:58:19.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 shutdown_connections 2023-10-18T17:58:19.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f29cc05a510 0x7f29cc05c9d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f29e0149660 0x7f29e00b8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e0149280 0x7f29e0144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 --2- 172.21.15.72:0/2209452647 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e00c0fb0 0x7f29e0147a40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 >> 172.21.15.72:0/2209452647 conn(0x7f29e0067510 msgr2=0x7f29e0069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:19.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 shutdown_connections 2023-10-18T17:58:19.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.190+0000 7f29e5546700 1 -- 172.21.15.72:0/2209452647 wait complete. 2023-10-18T17:58:19.199 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 3 2023-10-18T17:58:19.214 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:19.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.366+0000 7ff5015a9700 1 Processor -- start 2023-10-18T17:58:19.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.366+0000 7ff5015a9700 1 -- start start 2023-10-18T17:58:19.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff5015a9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 0x7ff4fc0bb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff5015a9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff4fc0660c0 0x7ff4fc14e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff5015a9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4fc14e450 con 0x7ff4fc0bf700 2023-10-18T17:58:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff5015a9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4fc0b95f0 con 0x7ff4fc0660c0 2023-10-18T17:58:19.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff5015a9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc14de20 con 0x7ff4fc0bbf10 2023-10-18T17:58:19.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4f3fff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ff4fc0bf700 0x7ff4fc0beb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:39192/0 (socket says 172.21.15.72:39192) 2023-10-18T17:58:19.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4f3fff700 1 -- 172.21.15.72:0/1375832922 learned_addr learned my addr 172.21.15.72:0/1375832922 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:19.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4fb7fe700 1 --2- 172.21.15.72:0/1375832922 >> v2:172.21.15.179:3300/0 conn(0x7ff4fc0660c0 0x7ff4fc14e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4faffd700 1 --2- 172.21.15.72:0/1375832922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 0x7ff4fc0bb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4fa7fc700 1 -- 172.21.15.72:0/1375832922 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805354915 0 0) 0x7ff4fc14e450 con 0x7ff4fc0bf700 2023-10-18T17:58:19.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4fb7fe700 1 -- 172.21.15.72:0/1375832922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 msgr2=0x7ff4fc0bb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:19.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4fb7fe700 1 --2- 172.21.15.72:0/1375832922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 0x7ff4fc0bb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4fb7fe700 1 -- 172.21.15.72:0/1375832922 >> v1:172.21.15.72:6789/0 conn(0x7ff4fc0bf700 legacy=0x7ff4fc0beb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4fb7fe700 1 -- 172.21.15.72:0/1375832922 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc0b9890 con 0x7ff4fc0660c0 2023-10-18T17:58:19.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.370+0000 7ff4fb7fe700 1 --2- 172.21.15.72:0/1375832922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4fc0660c0 0x7ff4fc14e8e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff4f4009e60 tx=0x7ff4f4009780).ready entity=mon.1 client_cookie=883ec260ff9e2a56 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:19.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff4fa7fc700 1 -- 172.21.15.72:0/1375832922 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f400e070 con 0x7ff4fc0660c0 2023-10-18T17:58:19.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff4fa7fc700 1 -- 172.21.15.72:0/1375832922 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f400bb60 con 0x7ff4fc0660c0 2023-10-18T17:58:19.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff4fa7fc700 1 -- 172.21.15.72:0/1375832922 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4012410 con 0x7ff4fc0660c0 2023-10-18T17:58:19.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 -- 172.21.15.72:0/1375832922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4fc0660c0 msgr2=0x7ff4fc14e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 --2- 172.21.15.72:0/1375832922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4fc0660c0 0x7ff4fc14e8e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff4f4009e60 tx=0x7ff4f4009780).stop 2023-10-18T17:58:19.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 -- 172.21.15.72:0/1375832922 shutdown_connections 2023-10-18T17:58:19.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 --2- 172.21.15.72:0/1375832922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4fc0660c0 0x7ff4fc14e8e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 --2- 172.21.15.72:0/1375832922 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 0x7ff4fc0bb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 -- 172.21.15.72:0/1375832922 >> 172.21.15.72:0/1375832922 conn(0x7ff4fc0620d0 msgr2=0x7ff4fc0624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:19.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 -- 172.21.15.72:0/1375832922 shutdown_connections 2023-10-18T17:58:19.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 -- 172.21.15.72:0/1375832922 wait complete. 2023-10-18T17:58:19.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 Processor -- start 2023-10-18T17:58:19.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.374+0000 7ff5015a9700 1 -- start start 2023-10-18T17:58:19.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff5015a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4fc0660c0 0x7ff4fc068060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff5015a9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 0x7ff4fc0b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff5015a9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4fc068550 0x7ff4fc0b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff5015a9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc149420 con 0x7ff4fc068550 2023-10-18T17:58:19.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff5015a9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc160cb0 con 0x7ff4fc0660c0 2023-10-18T17:58:19.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff5015a9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc0b7f70 con 0x7ff4fc0bbf10 2023-10-18T17:58:19.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4fb7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4fc068550 0x7ff4fc0b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4fb7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4fc068550 0x7ff4fc0b7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49378/0 (socket says 172.21.15.72:49378) 2023-10-18T17:58:19.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4fb7fe700 1 -- 172.21.15.72:0/3824050088 learned_addr learned my addr 172.21.15.72:0/3824050088 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:19.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4faffd700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4fc0660c0 0x7ff4fc068060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4f3fff700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 0x7ff4fc0b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4fb7fe700 1 -- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 msgr2=0x7ff4fc0b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4fb7fe700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 0x7ff4fc0b7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4fb7fe700 1 -- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4fc0660c0 msgr2=0x7ff4fc068060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4fb7fe700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4fc0660c0 0x7ff4fc068060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4fb7fe700 1 -- 172.21.15.72:0/3824050088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f4009a20 con 0x7ff4fc068550 2023-10-18T17:58:19.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4fb7fe700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4fc068550 0x7ff4fc0b7a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff4f400c000 tx=0x7ff4f4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:19.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4f2ffd700 1 -- 172.21.15.72:0/3824050088 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4012410 con 0x7ff4fc068550 2023-10-18T17:58:19.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4f2ffd700 1 -- 172.21.15.72:0/3824050088 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f401d630 con 0x7ff4fc068550 2023-10-18T17:58:19.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff4f2ffd700 1 -- 172.21.15.72:0/3824050088 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f401d9f0 con 0x7ff4fc068550 2023-10-18T17:58:19.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4fc15fe50 con 0x7ff4fc068550 2023-10-18T17:58:19.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.378+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4fc160410 con 0x7ff4fc068550 2023-10-18T17:58:19.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.382+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4fc0bac60 con 0x7ff4fc068550 2023-10-18T17:58:19.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.382+0000 7ff4f2ffd700 1 -- 172.21.15.72:0/3824050088 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff4f4003040 con 0x7ff4fc068550 2023-10-18T17:58:19.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.382+0000 7ff4f2ffd700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff4dc05a510 0x7ff4dc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.382+0000 7ff4f2ffd700 1 -- 172.21.15.72:0/3824050088 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff4f40762a0 con 0x7ff4fc068550 2023-10-18T17:58:19.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.386+0000 7ff4faffd700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff4dc05a510 0x7ff4dc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.392 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.386+0000 7ff4faffd700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff4dc05a510 0x7ff4dc05c9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff4ec009c30 tx=0x7ff4ec009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:19.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.390+0000 7ff4f2ffd700 1 -- 172.21.15.72:0/3824050088 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ff4f4049dd0 con 0x7ff4fc068550 2023-10-18T17:58:19.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.550+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4fc06c190 con 0x7ff4fc068550 2023-10-18T17:58:19.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff4f2ffd700 1 -- 172.21.15.72:0/3824050088 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13709 (secure 0 0 0) 0x7ff4f404d3f0 con 0x7ff4fc068550 2023-10-18T17:58:19.557 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:19.557 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":18,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:18.604045+0000","last_up_change":"2023-10-18T17:58:14.550597+0000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T17:58:13.627096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T17:58:15.732786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":33232},{"type":"v1","addr":"172.21.15.72:6810","nonce":33232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6813","nonce":33232},{"type":"v1","addr":"172.21.15.72:6815","nonce":33232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":33232},{"type":"v1","addr":"172.21.15.72:6826","nonce":33232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":33232},{"type":"v1","addr":"172.21.15.72:6820","nonce":33232}]},"public_addr":"172.21.15.72:6810/33232","cluster_addr":"172.21.15.72:6815/33232","heartbeat_back_addr":"172.21.15.72:6826/33232","heartbeat_front_addr":"172.21.15.72:6820/33232","state":["exists","up"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6809","nonce":33229},{"type":"v1","addr":"172.21.15.72:6812","nonce":33229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":33229},{"type":"v1","addr":"172.21.15.72:6819","nonce":33229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":33229},{"type":"v1","addr":"172.21.15.72:6831","nonce":33229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":33229},{"type":"v1","addr":"172.21.15.72:6825","nonce":33229}]},"public_addr":"172.21.15.72:6812/33229","cluster_addr":"172.21.15.72:6819/33229","heartbeat_back_addr":"172.21.15.72:6831/33229","heartbeat_front_addr":"172.21.15.72:6825/33229","state":["exists","up"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":33231},{"type":"v1","addr":"172.21.15.72:6801","nonce":33231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":33231},{"type":"v1","addr":"172.21.15.72:6803","nonce":33231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":33231},{"type":"v1","addr":"172.21.15.72:6807","nonce":33231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":33231},{"type":"v1","addr":"172.21.15.72:6805","nonce":33231}]},"public_addr":"172.21.15.72:6801/33231","cluster_addr":"172.21.15.72:6803/33231","heartbeat_back_addr":"172.21.15.72:6807/33231","heartbeat_front_addr":"172.21.15.72:6805/33231","state":["exists","up"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":33228},{"type":"v1","addr":"172.21.15.72:6814","nonce":33228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":33228},{"type":"v1","addr":"172.21.15.72:6821","nonce":33228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":33228},{"type":"v1","addr":"172.21.15.72:6830","nonce":33228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":33228},{"type":"v1","addr":"172.21.15.72:6827","nonce":33228}]},"public_addr":"172.21.15.72:6814/33228","cluster_addr":"172.21.15.72:6821/33228","heartbeat_back_addr":"172.21.15.72:6830/33228","heartbeat_front_addr":"172.21.15.72:6827/33228","state":["exists","up"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":34034},{"type":"v1","addr":"172.21.15.179:6809","nonce":34034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":34034},{"type":"v1","addr":"172.21.15.179:6813","nonce":34034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":34034},{"type":"v1","addr":"172.21.15.179:6821","nonce":34034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":34034},{"type":"v1","addr":"172.21.15.179:6817","nonce":34034}]},"public_addr":"172.21.15.179:6809/34034","cluster_addr":"172.21.15.179:6813/34034","heartbeat_back_addr":"172.21.15.179:6821/34034","heartbeat_front_addr":"172.21.15.179:6817/34034","state":["exists","up"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":34044},{"type":"v1","addr":"172.21.15.179:6801","nonce":34044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":34044},{"type":"v1","addr":"172.21.15.179:6803","nonce":34044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":34044},{"type":"v1","addr":"172.21.15.179:6807","nonce":34044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":34044},{"type":"v1","addr":"172.21.15.179:6805","nonce":34044}]},"public_addr":"172.21.15.179:6801/34044","cluster_addr":"172.21.15.179:6803/34044","heartbeat_back_addr":"172.21.15.179:6807/34044","heartbeat_front_addr":"172.21.15.179:6805/34044","state":["exists","up"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":34036},{"type":"v1","addr":"172.21.15.179:6825","nonce":34036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":34036},{"type":"v1","addr":"172.21.15.179:6827","nonce":34036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":34036},{"type":"v1","addr":"172.21.15.179:6831","nonce":34036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":34036},{"type":"v1","addr":"172.21.15.179:6829","nonce":34036}]},"public_addr":"172.21.15.179:6825/34036","cluster_addr":"172.21.15.179:6827/34036","heartbeat_back_addr":"172.21.15.179:6831/34036","heartbeat_front_addr":"172.21.15.179:6829/34036","state":["exists","up"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":34035},{"type":"v1","addr":"172.21.15.179:6812","nonce":34035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":34035},{"type":"v1","addr":"172.21.15.179:6816","nonce":34035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":34035},{"type":"v1","addr":"172.21.15.179:6823","nonce":34035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":34035},{"type":"v1","addr":"172.21.15.179:6820","nonce":34035}]},"public_addr":"172.21.15.179:6812/34035","cluster_addr":"172.21.15.179:6816/34035","heartbeat_back_addr":"172.21.15.179:6823/34035","heartbeat_front_addr":"172.21.15.179:6820/34035","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.697876+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.661119+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.579136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.792036+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.069244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:11.971339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.176699+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.197004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:19.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff4dc05a510 msgr2=0x7ff4dc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff4dc05a510 0x7ff4dc05c9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff4ec009c30 tx=0x7ff4ec009060).stop 2023-10-18T17:58:19.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4fc068550 msgr2=0x7ff4fc0b7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4fc068550 0x7ff4fc0b7a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff4f400c000 tx=0x7ff4f4009780).stop 2023-10-18T17:58:19.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 shutdown_connections 2023-10-18T17:58:19.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff4dc05a510 0x7ff4dc05c9d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4fc068550 0x7ff4fc0b7a80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff4fc0bbf10 0x7ff4fc0b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 --2- 172.21.15.72:0/3824050088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4fc0660c0 0x7ff4fc068060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 >> 172.21.15.72:0/3824050088 conn(0x7ff4fc0620d0 msgr2=0x7ff4fc0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:19.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 shutdown_connections 2023-10-18T17:58:19.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.554+0000 7ff5015a9700 1 -- 172.21.15.72:0/3824050088 wait complete. 2023-10-18T17:58:19.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-10-18T17:58:13.627096+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-18T17:58:15.732786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-18T17:58:19.583 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-10-18T17:58:19.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.754+0000 7f852f546700 1 Processor -- start 2023-10-18T17:58:19.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.754+0000 7f852f546700 1 -- start start 2023-10-18T17:58:19.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.754+0000 7f852f546700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8528065ce0 0x7f85280660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8528068960 0x7f85280bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85280bf6e0 con 0x7f852813d140 2023-10-18T17:58:19.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8528068d60 con 0x7f8528065ce0 2023-10-18T17:58:19.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8528154360 con 0x7f8528068960 2023-10-18T17:58:19.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852d2e2700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8528065ce0 0x7f85280660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852dae3700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f852813d140 0x7f852813d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:46832/0 (socket says 172.21.15.72:46832) 2023-10-18T17:58:19.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852dae3700 1 -- 172.21.15.72:0/573668728 learned_addr learned my addr 172.21.15.72:0/573668728 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:19.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852cae1700 1 --2- 172.21.15.72:0/573668728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8528068960 0x7f85280bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852d2e2700 1 -- 172.21.15.72:0/573668728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8528068960 msgr2=0x7f85280bca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:19.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852d2e2700 1 --2- 172.21.15.72:0/573668728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8528068960 0x7f85280bca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852d2e2700 1 -- 172.21.15.72:0/573668728 >> v1:172.21.15.179:6789/0 conn(0x7f852813d140 legacy=0x7f852813d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852d2e2700 1 -- 172.21.15.72:0/573668728 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8528154620 con 0x7f8528065ce0 2023-10-18T17:58:19.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852d2e2700 1 --2- 172.21.15.72:0/573668728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528065ce0 0x7f85280660c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f8524009760 tx=0x7f8524009060).ready entity=mon.1 client_cookie=44a36005715ddf18 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:19.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f851ffff700 1 -- 172.21.15.72:0/573668728 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400b590 con 0x7f8528065ce0 2023-10-18T17:58:19.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f851ffff700 1 -- 172.21.15.72:0/573668728 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852400b6f0 con 0x7f8528065ce0 2023-10-18T17:58:19.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f851ffff700 1 -- 172.21.15.72:0/573668728 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400ba70 con 0x7f8528065ce0 2023-10-18T17:58:19.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 -- 172.21.15.72:0/573668728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528065ce0 msgr2=0x7f85280660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 --2- 172.21.15.72:0/573668728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528065ce0 0x7f85280660c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f8524009760 tx=0x7f8524009060).stop 2023-10-18T17:58:19.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 -- 172.21.15.72:0/573668728 shutdown_connections 2023-10-18T17:58:19.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 --2- 172.21.15.72:0/573668728 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8528068960 0x7f85280bca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 --2- 172.21.15.72:0/573668728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528065ce0 0x7f85280660c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 -- 172.21.15.72:0/573668728 >> 172.21.15.72:0/573668728 conn(0x7f85280650d0 msgr2=0x7f852805dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:19.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 -- 172.21.15.72:0/573668728 shutdown_connections 2023-10-18T17:58:19.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.758+0000 7f852f546700 1 -- 172.21.15.72:0/573668728 wait complete. 2023-10-18T17:58:19.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.762+0000 7f852f546700 1 Processor -- start 2023-10-18T17:58:19.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.762+0000 7f852f546700 1 -- start start 2023-10-18T17:58:19.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852f546700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8528065ce0 0x7f85280c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852f546700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528068960 0x7f8528155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852f546700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852813d140 0x7f8528153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852f546700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85280c0030 con 0x7f8528065ce0 2023-10-18T17:58:19.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852f546700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8528135450 con 0x7f8528068960 2023-10-18T17:58:19.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852f546700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85281649a0 con 0x7f852813d140 2023-10-18T17:58:19.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852d2e2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8528065ce0 0x7f85280c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852cae1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528068960 0x7f8528155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852cae1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528068960 0x7f8528155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:34926/0 (socket says 172.21.15.72:34926) 2023-10-18T17:58:19.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852cae1700 1 -- 172.21.15.72:0/3312373777 learned_addr learned my addr 172.21.15.72:0/3312373777 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:19.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852d2e2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8528065ce0 0x7f85280c1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49384/0 (socket says 172.21.15.72:49384) 2023-10-18T17:58:19.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852dae3700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852813d140 0x7f8528153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852cae1700 1 -- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852813d140 msgr2=0x7f8528153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852cae1700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852813d140 0x7f8528153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852cae1700 1 -- 172.21.15.72:0/3312373777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8528065ce0 msgr2=0x7f85280c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852cae1700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8528065ce0 0x7f85280c1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852cae1700 1 -- 172.21.15.72:0/3312373777 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240092a0 con 0x7f8528068960 2023-10-18T17:58:19.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852d2e2700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8528065ce0 0x7f85280c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:19.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852cae1700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528068960 0x7f8528155900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f851800e930 tx=0x7f8518009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:19.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f851d7fa700 1 -- 172.21.15.72:0/3312373777 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518012600 con 0x7f8528068960 2023-10-18T17:58:19.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f851d7fa700 1 -- 172.21.15.72:0/3312373777 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8518012760 con 0x7f8528068960 2023-10-18T17:58:19.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f851d7fa700 1 -- 172.21.15.72:0/3312373777 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518012ae0 con 0x7f8528068960 2023-10-18T17:58:19.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8528165500 con 0x7f8528068960 2023-10-18T17:58:19.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.766+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8528164f40 con 0x7f8528068960 2023-10-18T17:58:19.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.770+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85280c01c0 con 0x7f8528068960 2023-10-18T17:58:19.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.770+0000 7f851d7fa700 1 -- 172.21.15.72:0/3312373777 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f851801b400 con 0x7f8528068960 2023-10-18T17:58:19.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.770+0000 7f851d7fa700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f850c05a510 0x7f850c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:19.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.770+0000 7f851d7fa700 1 -- 172.21.15.72:0/3312373777 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f8518074940 con 0x7f8528068960 2023-10-18T17:58:19.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.770+0000 7f852d2e2700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f850c05a510 0x7f850c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:19.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.774+0000 7f852d2e2700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f850c05a510 0x7f850c05c9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f852400b350 tx=0x7f8524017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:19.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.782+0000 7f851d7fa700 1 -- 172.21.15.72:0/3312373777 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f8518048350 con 0x7f8528068960 2023-10-18T17:58:19.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.954+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f852806c190 con 0x7f8528068960 2023-10-18T17:58:19.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.954+0000 7f851d7fa700 1 -- 172.21.15.72:0/3312373777 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f851804b970 con 0x7f8528068960 2023-10-18T17:58:19.959 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 1 2023-10-18T17:58:19.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f850c05a510 msgr2=0x7f850c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f850c05a510 0x7f850c05c9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f852400b350 tx=0x7f8524017000).stop 2023-10-18T17:58:19.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528068960 msgr2=0x7f8528155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:19.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528068960 0x7f8528155900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f851800e930 tx=0x7f8518009060).stop 2023-10-18T17:58:19.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 shutdown_connections 2023-10-18T17:58:19.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f850c05a510 0x7f850c05c9d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852813d140 0x7f8528153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8528068960 0x7f8528155900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 --2- 172.21.15.72:0/3312373777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8528065ce0 0x7f85280c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:19.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 >> 172.21.15.72:0/3312373777 conn(0x7f85280650d0 msgr2=0x7f8528157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:19.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 shutdown_connections 2023-10-18T17:58:19.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:19.958+0000 7f852f546700 1 -- 172.21.15.72:0/3312373777 wait complete. 2023-10-18T17:58:19.983 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-18T17:58:20.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa106a1b700 1 Processor -- start 2023-10-18T17:58:20.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa106a1b700 1 -- start start 2023-10-18T17:58:20.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa106a1b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa1000be120 0x7fa100150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa106a1b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 0x7fa1000bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa106a1b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa100150fa0 con 0x7fa1000bdd40 2023-10-18T17:58:20.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa106a1b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10014cad0 con 0x7fa1000be120 2023-10-18T17:58:20.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa106a1b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10014cce0 con 0x7fa1000ba840 2023-10-18T17:58:20.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa0fffff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fa1000bdd40 0x7fa1000bea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50496/0 (socket says 172.21.15.72:50496) 2023-10-18T17:58:20.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa0fffff700 1 -- 172.21.15.72:0/4021221181 learned_addr learned my addr 172.21.15.72:0/4021221181 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:20.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa104fb8700 1 --2- 172.21.15.72:0/4021221181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 0x7fa1000bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.150+0000 7fa0f77fe700 1 --2- 172.21.15.72:0/4021221181 >> v2:172.21.15.72:3300/0 conn(0x7fa1000be120 0x7fa100150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa0ff7fe700 1 -- 172.21.15.72:0/4021221181 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051541534 0 0) 0x7fa100150fa0 con 0x7fa1000bdd40 2023-10-18T17:58:20.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa0ff7fe700 1 -- 172.21.15.72:0/4021221181 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0ec0033f0 con 0x7fa1000bdd40 2023-10-18T17:58:20.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa104fb8700 1 -- 172.21.15.72:0/4021221181 >> v2:172.21.15.72:3300/0 conn(0x7fa1000be120 msgr2=0x7fa100150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:20.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa104fb8700 1 --2- 172.21.15.72:0/4021221181 >> v2:172.21.15.72:3300/0 conn(0x7fa1000be120 0x7fa100150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa104fb8700 1 -- 172.21.15.72:0/4021221181 >> v1:172.21.15.72:6789/0 conn(0x7fa1000bdd40 legacy=0x7fa1000bea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa104fb8700 1 -- 172.21.15.72:0/4021221181 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10014cfc0 con 0x7fa1000ba840 2023-10-18T17:58:20.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa104fb8700 1 --2- 172.21.15.72:0/4021221181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 0x7fa1000bacb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fa0f8009e60 tx=0x7fa0f8009780).ready entity=mon.2 client_cookie=eeade8637c33de39 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:20.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa0ff7fe700 1 -- 172.21.15.72:0/4021221181 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f800bb40 con 0x7fa1000ba840 2023-10-18T17:58:20.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa0ff7fe700 1 -- 172.21.15.72:0/4021221181 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f800bca0 con 0x7fa1000ba840 2023-10-18T17:58:20.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa0ff7fe700 1 -- 172.21.15.72:0/4021221181 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8011690 con 0x7fa1000ba840 2023-10-18T17:58:20.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa106a1b700 1 -- 172.21.15.72:0/4021221181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 msgr2=0x7fa1000bacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa106a1b700 1 --2- 172.21.15.72:0/4021221181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 0x7fa1000bacb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fa0f8009e60 tx=0x7fa0f8009780).stop 2023-10-18T17:58:20.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa106a1b700 1 -- 172.21.15.72:0/4021221181 shutdown_connections 2023-10-18T17:58:20.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa106a1b700 1 --2- 172.21.15.72:0/4021221181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 0x7fa1000bacb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa106a1b700 1 --2- 172.21.15.72:0/4021221181 >> v2:172.21.15.72:3300/0 conn(0x7fa1000be120 0x7fa100150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa106a1b700 1 -- 172.21.15.72:0/4021221181 >> 172.21.15.72:0/4021221181 conn(0x7fa1000650d0 msgr2=0x7fa10005dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:20.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa106a1b700 1 -- 172.21.15.72:0/4021221181 shutdown_connections 2023-10-18T17:58:20.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.154+0000 7fa106a1b700 1 -- 172.21.15.72:0/4021221181 wait complete. 2023-10-18T17:58:20.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.158+0000 7fa106a1b700 1 Processor -- start 2023-10-18T17:58:20.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.158+0000 7fa106a1b700 1 -- start start 2023-10-18T17:58:20.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa106a1b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 0x7fa1001607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa106a1b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1000bdd40 0x7fa1001450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa106a1b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1000be120 0x7fa100145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa106a1b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa100160c90 con 0x7fa1000be120 2023-10-18T17:58:20.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa106a1b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1000bb5d0 con 0x7fa1000bdd40 2023-10-18T17:58:20.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa106a1b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa100144c10 con 0x7fa1000ba840 2023-10-18T17:58:20.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa104fb8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1000be120 0x7fa100145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa0fffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 0x7fa1001607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa0f77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1000bdd40 0x7fa1001450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa104fb8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1000be120 0x7fa100145590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58658/0 (socket says 172.21.15.72:58658) 2023-10-18T17:58:20.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa104fb8700 1 -- 172.21.15.72:0/3507188565 learned_addr learned my addr 172.21.15.72:0/3507188565 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:20.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa104fb8700 1 -- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 msgr2=0x7fa1001607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa104fb8700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 0x7fa1001607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa104fb8700 1 -- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1000bdd40 msgr2=0x7fa1001450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa104fb8700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1000bdd40 0x7fa1001450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa104fb8700 1 -- 172.21.15.72:0/3507188565 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f8009a20 con 0x7fa1000be120 2023-10-18T17:58:20.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa104fb8700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1000be120 0x7fa100145590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa0f800b610 tx=0x7fa0f8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:20.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa0fcff9700 1 -- 172.21.15.72:0/3507188565 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f801a630 con 0x7fa1000be120 2023-10-18T17:58:20.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa0fcff9700 1 -- 172.21.15.72:0/3507188565 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f801a790 con 0x7fa1000be120 2023-10-18T17:58:20.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa0fcff9700 1 -- 172.21.15.72:0/3507188565 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f801ab50 con 0x7fa1000be120 2023-10-18T17:58:20.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10014b340 con 0x7fa1000be120 2023-10-18T17:58:20.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1001401f0 con 0x7fa1000be120 2023-10-18T17:58:20.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.162+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10006c190 con 0x7fa1000be120 2023-10-18T17:58:20.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.166+0000 7fa0fcff9700 1 -- 172.21.15.72:0/3507188565 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa0f8003040 con 0x7fa1000be120 2023-10-18T17:58:20.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.166+0000 7fa0fcff9700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa0e005a510 0x7fa0e005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.166+0000 7fa0fcff9700 1 -- 172.21.15.72:0/3507188565 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa0f8076220 con 0x7fa1000be120 2023-10-18T17:58:20.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.166+0000 7fa0fffff700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa0e005a510 0x7fa0e005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.166+0000 7fa0fffff700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa0e005a510 0x7fa0e005c9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa0f0005750 tx=0x7fa0f0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:20.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.174+0000 7fa0fcff9700 1 -- 172.21.15.72:0/3507188565 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fa0f8049d50 con 0x7fa1000be120 2023-10-18T17:58:20.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.330+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa1000b9260 con 0x7fa1000be120 2023-10-18T17:58:20.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.330+0000 7fa0fcff9700 1 -- 172.21.15.72:0/3507188565 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa0f80290a0 con 0x7fa1000be120 2023-10-18T17:58:20.336 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 8 2023-10-18T17:58:20.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa0e005a510 msgr2=0x7fa0e005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa0e005a510 0x7fa0e005c9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa0f0005750 tx=0x7fa0f0009040).stop 2023-10-18T17:58:20.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1000be120 msgr2=0x7fa100145590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1000be120 0x7fa100145590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa0f800b610 tx=0x7fa0f8009780).stop 2023-10-18T17:58:20.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 shutdown_connections 2023-10-18T17:58:20.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa0e005a510 0x7fa0e005c9d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1000be120 0x7fa100145590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1000bdd40 0x7fa1001450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 --2- 172.21.15.72:0/3507188565 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa1000ba840 0x7fa1001607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 >> 172.21.15.72:0/3507188565 conn(0x7fa1000650d0 msgr2=0x7fa100151250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:20.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 shutdown_connections 2023-10-18T17:58:20.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.334+0000 7fa106a1b700 1 -- 172.21.15.72:0/3507188565 wait complete. 2023-10-18T17:58:20.357 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T17:58:20.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ae08f700 1 Processor -- start 2023-10-18T17:58:20.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ae08f700 1 -- start start 2023-10-18T17:58:20.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ae08f700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f98a80b56b0 0x7f98a8065760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ae08f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98a81580f0 0x7f98a80bc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ae08f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a80bd180 con 0x7f98a80bcd70 2023-10-18T17:58:20.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ae08f700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a81541d0 con 0x7f98a80b56b0 2023-10-18T17:58:20.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ae08f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a80be7b0 con 0x7f98a81580f0 2023-10-18T17:58:20.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ad08d700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f98a80b56b0 0x7f98a8065760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ad88e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f98a80bcd70 0x7f98a80bec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:60674/0 (socket says 172.21.15.72:60674) 2023-10-18T17:58:20.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ad88e700 1 -- 172.21.15.72:0/3748276053 learned_addr learned my addr 172.21.15.72:0/3748276053 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:20.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ac88c700 1 --2- 172.21.15.72:0/3748276053 >> v2:172.21.15.179:3300/0 conn(0x7f98a81580f0 0x7f98a80bc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98a7fff700 1 -- 172.21.15.72:0/3748276053 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529148200 0 0) 0x7f98a0001490 con 0x7f98a80bcd70 2023-10-18T17:58:20.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ad08d700 1 -- 172.21.15.72:0/3748276053 >> v2:172.21.15.179:3300/0 conn(0x7f98a81580f0 msgr2=0x7f98a80bc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:20.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.518+0000 7f98ad08d700 1 --2- 172.21.15.72:0/3748276053 >> v2:172.21.15.179:3300/0 conn(0x7f98a81580f0 0x7f98a80bc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ad08d700 1 -- 172.21.15.72:0/3748276053 >> v1:172.21.15.179:6789/0 conn(0x7f98a80bcd70 legacy=0x7f98a80bec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ad08d700 1 -- 172.21.15.72:0/3748276053 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a8158540 con 0x7f98a80b56b0 2023-10-18T17:58:20.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ad08d700 1 --2- 172.21.15.72:0/3748276053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80b56b0 0x7f98a8065760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f98a80bbd80 tx=0x7f9898009060).ready entity=mon.0 client_cookie=8023888cb927ad18 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:20.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98a7fff700 1 -- 172.21.15.72:0/3748276053 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800f510 con 0x7f98a80b56b0 2023-10-18T17:58:20.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98a7fff700 1 -- 172.21.15.72:0/3748276053 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989800f670 con 0x7f98a80b56b0 2023-10-18T17:58:20.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98a7fff700 1 -- 172.21.15.72:0/3748276053 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800f9f0 con 0x7f98a80b56b0 2023-10-18T17:58:20.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ae08f700 1 -- 172.21.15.72:0/3748276053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80b56b0 msgr2=0x7f98a8065760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ae08f700 1 --2- 172.21.15.72:0/3748276053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80b56b0 0x7f98a8065760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f98a80bbd80 tx=0x7f9898009060).stop 2023-10-18T17:58:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ae08f700 1 -- 172.21.15.72:0/3748276053 shutdown_connections 2023-10-18T17:58:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ae08f700 1 --2- 172.21.15.72:0/3748276053 >> v2:172.21.15.179:3300/0 conn(0x7f98a81580f0 0x7f98a80bc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ae08f700 1 --2- 172.21.15.72:0/3748276053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80b56b0 0x7f98a8065760 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ae08f700 1 -- 172.21.15.72:0/3748276053 >> 172.21.15.72:0/3748276053 conn(0x7f98a80645a0 msgr2=0x7f98a80649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:20.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ae08f700 1 -- 172.21.15.72:0/3748276053 shutdown_connections 2023-10-18T17:58:20.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.522+0000 7f98ae08f700 1 -- 172.21.15.72:0/3748276053 wait complete. 2023-10-18T17:58:20.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.526+0000 7f98ae08f700 1 Processor -- start 2023-10-18T17:58:20.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.526+0000 7f98ae08f700 1 -- start start 2023-10-18T17:58:20.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ae08f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a80b56b0 0x7f98a80b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ae08f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80bcd70 0x7f98a8156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ae08f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a81580f0 0x7f98a8156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ae08f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80b7d40 con 0x7f98a80bcd70 2023-10-18T17:58:20.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ae08f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8151100 con 0x7f98a81580f0 2023-10-18T17:58:20.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ae08f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80bb1a0 con 0x7f98a80b56b0 2023-10-18T17:58:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ad08d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a80b56b0 0x7f98a80b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ac88c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80bcd70 0x7f98a8156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ad88e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a81580f0 0x7f98a8156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ad08d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a80b56b0 0x7f98a80b9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:40692/0 (socket says 172.21.15.72:40692) 2023-10-18T17:58:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ad08d700 1 -- 172.21.15.72:0/2987866803 learned_addr learned my addr 172.21.15.72:0/2987866803 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ac88c700 1 -- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a80b56b0 msgr2=0x7f98a80b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ac88c700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a80b56b0 0x7f98a80b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ac88c700 1 -- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a81580f0 msgr2=0x7f98a8156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ac88c700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a81580f0 0x7f98a8156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ac88c700 1 -- 172.21.15.72:0/2987866803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98980092a0 con 0x7f98a80bcd70 2023-10-18T17:58:20.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ac88c700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80bcd70 0x7f98a8156790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f989c00e900 tx=0x7f989c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:20.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98a57fa700 1 -- 172.21.15.72:0/2987866803 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c012600 con 0x7f98a80bcd70 2023-10-18T17:58:20.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98a57fa700 1 -- 172.21.15.72:0/2987866803 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c012760 con 0x7f98a80bcd70 2023-10-18T17:58:20.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a8157170 con 0x7f98a80bcd70 2023-10-18T17:58:20.539 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98a57fa700 1 -- 172.21.15.72:0/2987866803 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c012ae0 con 0x7f98a80bcd70 2023-10-18T17:58:20.539 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.530+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a81676f0 con 0x7f98a80bcd70 2023-10-18T17:58:20.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.534+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a8167910 con 0x7f98a80bcd70 2023-10-18T17:58:20.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.534+0000 7f98a57fa700 1 -- 172.21.15.72:0/2987866803 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f989c01b400 con 0x7f98a80bcd70 2023-10-18T17:58:20.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.534+0000 7f98a57fa700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989405a510 0x7f989405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.534+0000 7f98a57fa700 1 -- 172.21.15.72:0/2987866803 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f989c074920 con 0x7f98a80bcd70 2023-10-18T17:58:20.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.538+0000 7f98ad08d700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989405a510 0x7f989405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.538+0000 7f98ad08d700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989405a510 0x7f989405c9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f98a8066f00 tx=0x7f9898017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:20.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.542+0000 7f98a57fa700 1 -- 172.21.15.72:0/2987866803 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f989c048330 con 0x7f98a80bcd70 2023-10-18T17:58:20.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.710+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f98a805eba0 con 0x7f98a80bcd70 2023-10-18T17:58:20.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.710+0000 7f98a57fa700 1 -- 172.21.15.72:0/2987866803 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7f989c04b950 con 0x7f98a80bcd70 2023-10-18T17:58:20.717 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:20.717 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2023-10-18T17:58:20.717 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-10-18T17:58:20.718 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 1, 2023-10-18T17:58:20.718 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": "device_health_metrics" 2023-10-18T17:58:20.719 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-10-18T17:58:20.719 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-10-18T17:58:20.719 INFO:teuthology.orchestra.run.smithi072.stdout: "poolnum": 2, 2023-10-18T17:58:20.720 INFO:teuthology.orchestra.run.smithi072.stdout: "poolname": "rbd" 2023-10-18T17:58:20.720 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-10-18T17:58:20.720 INFO:teuthology.orchestra.run.smithi072.stdout:] 2023-10-18T17:58:20.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989405a510 msgr2=0x7f989405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989405a510 0x7f989405c9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f98a8066f00 tx=0x7f9898017000).stop 2023-10-18T17:58:20.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80bcd70 msgr2=0x7f98a8156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80bcd70 0x7f98a8156790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f989c00e900 tx=0x7f989c009060).stop 2023-10-18T17:58:20.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 shutdown_connections 2023-10-18T17:58:20.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989405a510 0x7f989405c9d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98a81580f0 0x7f98a8156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98a80bcd70 0x7f98a8156790 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 --2- 172.21.15.72:0/2987866803 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98a80b56b0 0x7f98a80b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 >> 172.21.15.72:0/2987866803 conn(0x7f98a80645a0 msgr2=0x7f98a80bb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:20.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 shutdown_connections 2023-10-18T17:58:20.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.714+0000 7f98ae08f700 1 -- 172.21.15.72:0/2987866803 wait complete. 2023-10-18T17:58:20.737 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T17:58:20.738 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-10-18T17:58:20.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.910+0000 7ff08e08f700 1 Processor -- start 2023-10-18T17:58:20.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.910+0000 7ff08e08f700 1 -- start start 2023-10-18T17:58:20.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08e08f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0880c11f0 0x7ff0880c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08e08f700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff088065670 0x7ff088065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08e08f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08814ffb0 con 0x7ff0880c23e0 2023-10-18T17:58:20.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08e08f700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff088150df0 con 0x7ff088065670 2023-10-18T17:58:20.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08e08f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff088150270 con 0x7ff0880c11f0 2023-10-18T17:58:20.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08d88e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff088065670 0x7ff088065ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08c88c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0880c11f0 0x7ff0880c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08d88e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff088065670 0x7ff088065ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58674/0 (socket says 172.21.15.72:58674) 2023-10-18T17:58:20.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08d88e700 1 -- 172.21.15.72:0/3252401950 learned_addr learned my addr 172.21.15.72:0/3252401950 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:20.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08d08d700 1 --1- 172.21.15.72:0/3252401950 >> v1:172.21.15.179:6789/0 conn(0x7ff0880c23e0 0x7ff0880664a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:60690/0 (socket says 172.21.15.72:60690) 2023-10-18T17:58:20.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08c88c700 1 -- 172.21.15.72:0/3252401950 >> v2:172.21.15.72:3300/0 conn(0x7ff088065670 msgr2=0x7ff088065ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:20.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08c88c700 1 --2- 172.21.15.72:0/3252401950 >> v2:172.21.15.72:3300/0 conn(0x7ff088065670 0x7ff088065ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08c88c700 1 -- 172.21.15.72:0/3252401950 >> v1:172.21.15.179:6789/0 conn(0x7ff0880c23e0 legacy=0x7ff0880664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08c88c700 1 -- 172.21.15.72:0/3252401950 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff088150550 con 0x7ff0880c11f0 2023-10-18T17:58:20.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff08d88e700 1 --2- 172.21.15.72:0/3252401950 >> v2:172.21.15.72:3300/0 conn(0x7ff088065670 0x7ff088065ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:20.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.914+0000 7ff07ffff700 1 -- 172.21.15.72:0/3252401950 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358808981 0 0) 0x7ff08814ffb0 con 0x7ff0880c23e0 2023-10-18T17:58:20.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08c88c700 1 --2- 172.21.15.72:0/3252401950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 0x7ff0880c27c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff088064bd0 tx=0x7ff078009060).ready entity=mon.1 client_cookie=1973388ad13dd4f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:20.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff07ffff700 1 -- 172.21.15.72:0/3252401950 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07800f510 con 0x7ff0880c11f0 2023-10-18T17:58:20.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff07ffff700 1 -- 172.21.15.72:0/3252401950 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07800f670 con 0x7ff0880c11f0 2023-10-18T17:58:20.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff07ffff700 1 -- 172.21.15.72:0/3252401950 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07800f9f0 con 0x7ff0880c11f0 2023-10-18T17:58:20.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08e08f700 1 -- 172.21.15.72:0/3252401950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 msgr2=0x7ff0880c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08e08f700 1 --2- 172.21.15.72:0/3252401950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 0x7ff0880c27c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff088064bd0 tx=0x7ff078009060).stop 2023-10-18T17:58:20.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08e08f700 1 -- 172.21.15.72:0/3252401950 shutdown_connections 2023-10-18T17:58:20.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08e08f700 1 --2- 172.21.15.72:0/3252401950 >> v2:172.21.15.72:3300/0 conn(0x7ff088065670 0x7ff088065ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08e08f700 1 --2- 172.21.15.72:0/3252401950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 0x7ff0880c27c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08e08f700 1 -- 172.21.15.72:0/3252401950 >> 172.21.15.72:0/3252401950 conn(0x7ff0880620d0 msgr2=0x7ff0880624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:20.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08e08f700 1 -- 172.21.15.72:0/3252401950 shutdown_connections 2023-10-18T17:58:20.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08e08f700 1 -- 172.21.15.72:0/3252401950 wait complete. 2023-10-18T17:58:20.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.918+0000 7ff08e08f700 1 Processor -- start 2023-10-18T17:58:20.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08e08f700 1 -- start start 2023-10-18T17:58:20.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08e08f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff088065670 0x7ff0880c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08e08f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 0x7ff088062650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08e08f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0880c23e0 0x7ff088161530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08e08f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880b3a90 con 0x7ff088065670 2023-10-18T17:58:20.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08e08f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff088160e40 con 0x7ff0880c11f0 2023-10-18T17:58:20.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08e08f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0881637d0 con 0x7ff0880c23e0 2023-10-18T17:58:20.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08c88c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 0x7ff088062650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08c88c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 0x7ff088062650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51050/0 (socket says 172.21.15.72:51050) 2023-10-18T17:58:20.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08c88c700 1 -- 172.21.15.72:0/580337017 learned_addr learned my addr 172.21.15.72:0/580337017 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:20.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08d88e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0880c23e0 0x7ff088161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08d08d700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff088065670 0x7ff0880c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08c88c700 1 -- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0880c23e0 msgr2=0x7ff088161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08c88c700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0880c23e0 0x7ff088161530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08c88c700 1 -- 172.21.15.72:0/580337017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff088065670 msgr2=0x7ff0880c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:20.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08c88c700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff088065670 0x7ff0880c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:20.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08c88c700 1 -- 172.21.15.72:0/580337017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080009ae0 con 0x7ff0880c11f0 2023-10-18T17:58:20.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08c88c700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 0x7ff088062650 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff08805e980 tx=0x7ff078017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:20.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff07d7fa700 1 -- 172.21.15.72:0/580337017 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07800f7e0 con 0x7ff0880c11f0 2023-10-18T17:58:20.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff07d7fa700 1 -- 172.21.15.72:0/580337017 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07800f940 con 0x7ff0880c11f0 2023-10-18T17:58:20.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff07d7fa700 1 -- 172.21.15.72:0/580337017 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07801b660 con 0x7ff0880c11f0 2023-10-18T17:58:20.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.922+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0780092a0 con 0x7ff0880c11f0 2023-10-18T17:58:20.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.926+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff088163d90 con 0x7ff0880c11f0 2023-10-18T17:58:20.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.926+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff088068880 con 0x7ff0880c11f0 2023-10-18T17:58:20.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.926+0000 7ff07d7fa700 1 -- 172.21.15.72:0/580337017 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff07801b9d0 con 0x7ff0880c11f0 2023-10-18T17:58:20.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.930+0000 7ff07d7fa700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff06c05a540 0x7ff06c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:20.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.930+0000 7ff07d7fa700 1 -- 172.21.15.72:0/580337017 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff078075c80 con 0x7ff0880c11f0 2023-10-18T17:58:20.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.930+0000 7ff08d08d700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff06c05a540 0x7ff06c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:20.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.930+0000 7ff08d08d700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff06c05a540 0x7ff06c05ca00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff084007fd0 tx=0x7ff084007d70).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:20.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:20.938+0000 7ff07d7fa700 1 -- 172.21.15.72:0/580337017 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ff088068880 con 0x7ff0880c11f0 2023-10-18T17:58:21.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.098+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7ff088068880 con 0x7ff0880c11f0 2023-10-18T17:58:21.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.646+0000 7ff07d7fa700 1 -- 172.21.15.72:0/580337017 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 152+0+0 (secure 0 0 0) 0x7ff088068880 con 0x7ff0880c11f0 2023-10-18T17:58:21.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.650+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff06c05a540 msgr2=0x7ff06c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:21.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.650+0000 7ff08e08f700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff06c05a540 0x7ff06c05ca00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff084007fd0 tx=0x7ff084007d70).stop 2023-10-18T17:58:21.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.650+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 msgr2=0x7ff088062650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:21.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.650+0000 7ff08e08f700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 0x7ff088062650 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff08805e980 tx=0x7ff078017000).stop 2023-10-18T17:58:21.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.654+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 shutdown_connections 2023-10-18T17:58:21.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.654+0000 7ff08e08f700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff06c05a540 0x7ff06c05ca00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:21.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.654+0000 7ff08e08f700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff0880c23e0 0x7ff088161530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:21.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.654+0000 7ff08e08f700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0880c11f0 0x7ff088062650 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:21.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.654+0000 7ff08e08f700 1 --2- 172.21.15.72:0/580337017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff088065670 0x7ff0880c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:21.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.654+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 >> 172.21.15.72:0/580337017 conn(0x7ff0880620d0 msgr2=0x7ff08806afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:21.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.654+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 shutdown_connections 2023-10-18T17:58:21.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.654+0000 7ff08e08f700 1 -- 172.21.15.72:0/580337017 wait complete. 2023-10-18T17:58:21.663 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_metadata' created 2023-10-18T17:58:21.679 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T17:58:21.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.846+0000 7f1c4065b700 1 Processor -- start 2023-10-18T17:58:21.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.846+0000 7f1c4065b700 1 -- start start 2023-10-18T17:58:21.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.850+0000 7f1c4065b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c38065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:21.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.850+0000 7f1c4065b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f1c38153cf0 0x7f1c38067fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:21.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.850+0000 7f1c4065b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c38150c60 con 0x7f1c38066030 2023-10-18T17:58:21.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.850+0000 7f1c4065b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c38068f10 con 0x7f1c38153cf0 2023-10-18T17:58:21.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.850+0000 7f1c4065b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38066470 con 0x7f1c380b58b0 2023-10-18T17:58:21.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.850+0000 7f1c3dbf6700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f1c38066030 0x7f1c381535e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50508/0 (socket says 172.21.15.72:50508) 2023-10-18T17:58:21.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.850+0000 7f1c3dbf6700 1 -- 172.21.15.72:0/829168019 learned_addr learned my addr 172.21.15.72:0/829168019 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:21.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.850+0000 7f1c3e3f7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c38065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:21.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.850+0000 7f1c3ebf8700 1 --2- 172.21.15.72:0/829168019 >> v2:172.21.15.72:3300/0 conn(0x7f1c38153cf0 0x7f1c38067fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:21.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c3e3f7700 1 -- 172.21.15.72:0/829168019 >> v2:172.21.15.72:3300/0 conn(0x7f1c38153cf0 msgr2=0x7f1c38067fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:21.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c3e3f7700 1 --2- 172.21.15.72:0/829168019 >> v2:172.21.15.72:3300/0 conn(0x7f1c38153cf0 0x7f1c38067fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:21.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c3e3f7700 1 -- 172.21.15.72:0/829168019 >> v1:172.21.15.72:6789/0 conn(0x7f1c38066030 legacy=0x7f1c381535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:21.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c3d3f5700 1 -- 172.21.15.72:0/829168019 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122969166 0 0) 0x7f1c38150c60 con 0x7f1c38066030 2023-10-18T17:58:21.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c3e3f7700 1 -- 172.21.15.72:0/829168019 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c380bf6e0 con 0x7f1c380b58b0 2023-10-18T17:58:21.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c3e3f7700 1 --2- 172.21.15.72:0/829168019 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c38065b40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f1c34009790 tx=0x7f1c34009060).ready entity=mon.2 client_cookie=67df099228602cfe server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:21.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c3d3f5700 1 -- 172.21.15.72:0/829168019 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400f510 con 0x7f1c380b58b0 2023-10-18T17:58:21.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c3d3f5700 1 -- 172.21.15.72:0/829168019 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3400f670 con 0x7f1c380b58b0 2023-10-18T17:58:21.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c3d3f5700 1 -- 172.21.15.72:0/829168019 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400f9f0 con 0x7f1c380b58b0 2023-10-18T17:58:21.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c4065b700 1 -- 172.21.15.72:0/829168019 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 msgr2=0x7f1c38065b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:21.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c4065b700 1 --2- 172.21.15.72:0/829168019 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c38065b40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f1c34009790 tx=0x7f1c34009060).stop 2023-10-18T17:58:21.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c4065b700 1 -- 172.21.15.72:0/829168019 shutdown_connections 2023-10-18T17:58:21.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c4065b700 1 --2- 172.21.15.72:0/829168019 >> v2:172.21.15.72:3300/0 conn(0x7f1c38153cf0 0x7f1c38067fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:21.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c4065b700 1 --2- 172.21.15.72:0/829168019 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c38065b40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:21.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c4065b700 1 -- 172.21.15.72:0/829168019 >> 172.21.15.72:0/829168019 conn(0x7f1c380650d0 msgr2=0x7f1c3805dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:21.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c4065b700 1 -- 172.21.15.72:0/829168019 shutdown_connections 2023-10-18T17:58:21.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.854+0000 7f1c4065b700 1 -- 172.21.15.72:0/829168019 wait complete. 2023-10-18T17:58:21.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c4065b700 1 Processor -- start 2023-10-18T17:58:21.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c4065b700 1 -- start start 2023-10-18T17:58:21.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c4065b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c38066030 0x7f1c380c1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:21.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c4065b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c380b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:21.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c4065b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c38153cf0 0x7f1c381633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:21.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c4065b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3814a440 con 0x7f1c38066030 2023-10-18T17:58:21.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c4065b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380b9820 con 0x7f1c38153cf0 2023-10-18T17:58:21.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c4065b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380b9520 con 0x7f1c380b58b0 2023-10-18T17:58:21.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3e3f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c38066030 0x7f1c380c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:21.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3ebf8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c38153cf0 0x7f1c381633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:21.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3e3f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c38066030 0x7f1c380c1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58698/0 (socket says 172.21.15.72:58698) 2023-10-18T17:58:21.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3e3f7700 1 -- 172.21.15.72:0/2651818738 learned_addr learned my addr 172.21.15.72:0/2651818738 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:21.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3ebf8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c38153cf0 0x7f1c381633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51064/0 (socket says 172.21.15.72:51064) 2023-10-18T17:58:21.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3dbf6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c380b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:21.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3e3f7700 1 -- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 msgr2=0x7f1c380b9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:21.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3e3f7700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c380b9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:21.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3e3f7700 1 -- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c38153cf0 msgr2=0x7f1c381633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:21.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3e3f7700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c38153cf0 0x7f1c381633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:21.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3e3f7700 1 -- 172.21.15.72:0/2651818738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c340092a0 con 0x7f1c38066030 2023-10-18T17:58:21.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3ebf8700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c38153cf0 0x7f1c381633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:21.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.858+0000 7f1c3dbf6700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c380b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:21.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.862+0000 7f1c3e3f7700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c38066030 0x7f1c380c1220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1c34006df0 tx=0x7f1c34017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:21.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.862+0000 7f1c2e7fc700 1 -- 172.21.15.72:0/2651818738 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400f7e0 con 0x7f1c38066030 2023-10-18T17:58:21.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.862+0000 7f1c2e7fc700 1 -- 172.21.15.72:0/2651818738 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3400f940 con 0x7f1c38066030 2023-10-18T17:58:21.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.862+0000 7f1c2e7fc700 1 -- 172.21.15.72:0/2651818738 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3401b660 con 0x7f1c38066030 2023-10-18T17:58:21.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.862+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c38160c50 con 0x7f1c38066030 2023-10-18T17:58:21.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.862+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c380baef0 con 0x7f1c38066030 2023-10-18T17:58:21.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.862+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3806c190 con 0x7f1c38066030 2023-10-18T17:58:21.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.862+0000 7f1c2e7fc700 1 -- 172.21.15.72:0/2651818738 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1c3401ba10 con 0x7f1c38066030 2023-10-18T17:58:21.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.866+0000 7f1c2e7fc700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1c1c05a510 0x7f1c1c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:21.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.866+0000 7f1c2e7fc700 1 -- 172.21.15.72:0/2651818738 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f1c340761d0 con 0x7f1c38066030 2023-10-18T17:58:21.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.866+0000 7f1c3dbf6700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1c1c05a510 0x7f1c1c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:21.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.866+0000 7f1c3dbf6700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1c1c05a510 0x7f1c1c05c9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f1c380bab00 tx=0x7f1c28007bb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:21.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:21.870+0000 7f1c2e7fc700 1 -- 172.21.15.72:0/2651818738 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f1c34049c70 con 0x7f1c38066030 2023-10-18T17:58:22.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.038+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1c380bf9a0 con 0x7f1c38066030 2023-10-18T17:58:22.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.654+0000 7f1c2e7fc700 1 -- 172.21.15.72:0/2651818738 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1c34028020 con 0x7f1c38066030 2023-10-18T17:58:22.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.658+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1c1c05a510 msgr2=0x7f1c1c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:22.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.658+0000 7f1c4065b700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1c1c05a510 0x7f1c1c05c9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f1c380bab00 tx=0x7f1c28007bb0).stop 2023-10-18T17:58:22.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.658+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c38066030 msgr2=0x7f1c380c1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:22.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.658+0000 7f1c4065b700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c38066030 0x7f1c380c1220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1c34006df0 tx=0x7f1c34017000).stop 2023-10-18T17:58:22.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.662+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 shutdown_connections 2023-10-18T17:58:22.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.662+0000 7f1c4065b700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1c1c05a510 0x7f1c1c05c9d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.662+0000 7f1c4065b700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c38153cf0 0x7f1c381633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.662+0000 7f1c4065b700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1c380b58b0 0x7f1c380b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.662+0000 7f1c4065b700 1 --2- 172.21.15.72:0/2651818738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c38066030 0x7f1c380c1220 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.662+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 >> 172.21.15.72:0/2651818738 conn(0x7f1c380650d0 msgr2=0x7f1c3814df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:22.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.666+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 shutdown_connections 2023-10-18T17:58:22.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.666+0000 7f1c4065b700 1 -- 172.21.15.72:0/2651818738 wait complete. 2023-10-18T17:58:22.681 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'cephfs_data' created 2023-10-18T17:58:22.704 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T17:58:22.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.862+0000 7ff60e69b700 1 Processor -- start 2023-10-18T17:58:22.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.862+0000 7ff60e69b700 1 -- start start 2023-10-18T17:58:22.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60e69b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080c4290 0x7ff6080c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:22.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60e69b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff6080bfd50 0x7ff608067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:22.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60e69b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff6080684e0 0x7ff6080c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:22.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60e69b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff608068c30 con 0x7ff6080bfd50 2023-10-18T17:58:22.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60e69b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6080c4b60 con 0x7ff6080684e0 2023-10-18T17:58:22.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60e69b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff608068960 con 0x7ff6080c4290 2023-10-18T17:58:22.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60cc38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff6080684e0 0x7ff6080c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:22.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff5fffff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff6080bfd50 0x7ff608067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:22.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60cc38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff6080684e0 0x7ff6080c0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51072/0 (socket says 172.21.15.72:51072) 2023-10-18T17:58:22.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60cc38700 1 -- 172.21.15.72:0/2187949503 learned_addr learned my addr 172.21.15.72:0/2187949503 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:22.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff607fff700 1 --2- 172.21.15.72:0/2187949503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080c4290 0x7ff6080c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:22.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff5fffff700 1 -- 172.21.15.72:0/2187949503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080c4290 msgr2=0x7ff6080c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:22.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff5fffff700 1 --2- 172.21.15.72:0/2187949503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080c4290 0x7ff6080c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff5fffff700 1 -- 172.21.15.72:0/2187949503 >> v2:172.21.15.179:3300/0 conn(0x7ff6080684e0 msgr2=0x7ff6080c0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:22.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff5fffff700 1 --2- 172.21.15.72:0/2187949503 >> v2:172.21.15.179:3300/0 conn(0x7ff6080684e0 0x7ff6080c0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff5fffff700 1 -- 172.21.15.72:0/2187949503 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608153720 con 0x7ff6080bfd50 2023-10-18T17:58:22.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff60cc38700 1 --2- 172.21.15.72:0/2187949503 >> v2:172.21.15.179:3300/0 conn(0x7ff6080684e0 0x7ff6080c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:22.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff5fffff700 1 --2- 172.21.15.72:0/2187949503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6080bfd50 0x7ff608067ff0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff608069c70 tx=0x7ff5f0009060).ready entity=mon.0 client_cookie=73384b171cb43814 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:22.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff6077fe700 1 -- 172.21.15.72:0/2187949503 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000d4a0 con 0x7ff6080bfd50 2023-10-18T17:58:22.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff6077fe700 1 -- 172.21.15.72:0/2187949503 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f000d600 con 0x7ff6080bfd50 2023-10-18T17:58:22.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.866+0000 7ff6077fe700 1 -- 172.21.15.72:0/2187949503 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000d980 con 0x7ff6080bfd50 2023-10-18T17:58:22.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 -- 172.21.15.72:0/2187949503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6080bfd50 msgr2=0x7ff608067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:22.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 --2- 172.21.15.72:0/2187949503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6080bfd50 0x7ff608067ff0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff608069c70 tx=0x7ff5f0009060).stop 2023-10-18T17:58:22.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 -- 172.21.15.72:0/2187949503 shutdown_connections 2023-10-18T17:58:22.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 --2- 172.21.15.72:0/2187949503 >> v2:172.21.15.179:3300/0 conn(0x7ff6080684e0 0x7ff6080c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 --2- 172.21.15.72:0/2187949503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6080bfd50 0x7ff608067ff0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 --2- 172.21.15.72:0/2187949503 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080c4290 0x7ff6080c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 -- 172.21.15.72:0/2187949503 >> 172.21.15.72:0/2187949503 conn(0x7ff6080667b0 msgr2=0x7ff608066bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:22.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 -- 172.21.15.72:0/2187949503 shutdown_connections 2023-10-18T17:58:22.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 -- 172.21.15.72:0/2187949503 wait complete. 2023-10-18T17:58:22.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 Processor -- start 2023-10-18T17:58:22.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 -- start start 2023-10-18T17:58:22.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080684e0 0x7ff608160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:22.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6080bfd50 0x7ff6081520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:22.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6080c4290 0x7ff608163df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:22.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6080a8ab0 con 0x7ff6080c4290 2023-10-18T17:58:22.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60814a5f0 con 0x7ff6080bfd50 2023-10-18T17:58:22.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.870+0000 7ff60e69b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff608161130 con 0x7ff6080684e0 2023-10-18T17:58:22.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6080bfd50 0x7ff6081520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:22.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff60cc38700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6080c4290 0x7ff608163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:22.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6080bfd50 0x7ff6081520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51074/0 (socket says 172.21.15.72:51074) 2023-10-18T17:58:22.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff5fffff700 1 -- 172.21.15.72:0/3922804067 learned_addr learned my addr 172.21.15.72:0/3922804067 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:22.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff607fff700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080684e0 0x7ff608160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:22.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff5fffff700 1 -- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080684e0 msgr2=0x7ff608160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:22.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff5fffff700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080684e0 0x7ff608160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff5fffff700 1 -- 172.21.15.72:0/3922804067 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6080c4290 msgr2=0x7ff608163df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:22.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff5fffff700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6080c4290 0x7ff608163df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:22.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff5fffff700 1 -- 172.21.15.72:0/3922804067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff600009ae0 con 0x7ff6080bfd50 2023-10-18T17:58:22.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff5fffff700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6080bfd50 0x7ff6081520e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7ff6080c1e10 tx=0x7ff5f0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:22.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff604ff9700 1 -- 172.21.15.72:0/3922804067 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000dd30 con 0x7ff6080bfd50 2023-10-18T17:58:22.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff604ff9700 1 -- 172.21.15.72:0/3922804067 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f000de90 con 0x7ff6080bfd50 2023-10-18T17:58:22.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff604ff9700 1 -- 172.21.15.72:0/3922804067 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000d9c0 con 0x7ff6080bfd50 2023-10-18T17:58:22.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f00092a0 con 0x7ff6080bfd50 2023-10-18T17:58:22.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6081638f0 con 0x7ff6080bfd50 2023-10-18T17:58:22.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.874+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60806c190 con 0x7ff6080bfd50 2023-10-18T17:58:22.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.878+0000 7ff604ff9700 1 -- 172.21.15.72:0/3922804067 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff5f00263b0 con 0x7ff6080bfd50 2023-10-18T17:58:22.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.878+0000 7ff604ff9700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff5e805a540 0x7ff5e805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:22.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.878+0000 7ff604ff9700 1 -- 172.21.15.72:0/3922804067 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6021+0+0 (secure 0 0 0) 0x7ff5f0075560 con 0x7ff6080bfd50 2023-10-18T17:58:22.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.878+0000 7ff607fff700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff5e805a540 0x7ff5e805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:22.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.878+0000 7ff607fff700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff5e805a540 0x7ff5e805ca00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff5f8009b90 tx=0x7ff5f8009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:22.893 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:22.890+0000 7ff604ff9700 1 -- 172.21.15.72:0/3922804067 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ff5f0048c80 con 0x7ff6080bfd50 2023-10-18T17:58:23.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.086+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff608063d60 con 0x7ff6080bfd50 2023-10-18T17:58:23.098 INFO:tasks.ceph.mds.a.smithi072.stderr:terminate called after throwing an instance of 'boost::wrapexcept' 2023-10-18T17:58:23.099 INFO:tasks.ceph.mds.a.smithi072.stderr: what(): boost::bad_get: failed value get using boost::get 2023-10-18T17:58:23.099 INFO:tasks.ceph.mds.a.smithi072.stderr:*** Caught signal (Aborted) ** 2023-10-18T17:58:23.099 INFO:tasks.ceph.mds.a.smithi072.stderr: in thread 7f968eb60700 thread_name:ms_dispatch 2023-10-18T17:58:23.103 INFO:tasks.ceph.mds.a.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:58:23.103 INFO:tasks.ceph.mds.a.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f9693db6420] 2023-10-18T17:58:23.103 INFO:tasks.ceph.mds.a.smithi072.stderr: 2: gsignal() 2023-10-18T17:58:23.103 INFO:tasks.ceph.mds.a.smithi072.stderr: 3: abort() 2023-10-18T17:58:23.104 INFO:tasks.ceph.mds.a.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f9693c5e911] 2023-10-18T17:58:23.104 INFO:tasks.ceph.mds.a.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f9693c6a38c] 2023-10-18T17:58:23.104 INFO:tasks.ceph.mds.a.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f9693c6a3f7] 2023-10-18T17:58:23.104 INFO:tasks.ceph.mds.a.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f9693c6a6a9] 2023-10-18T17:58:23.104 INFO:tasks.ceph.mds.a.smithi072.stderr: 8: ceph-mds(+0x185107) [0x56457536b107] 2023-10-18T17:58:23.105 INFO:tasks.ceph.mds.a.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x56457536b72d] 2023-10-18T17:58:23.105 INFO:tasks.ceph.mds.a.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5645753dd09f] 2023-10-18T17:58:23.105 INFO:tasks.ceph.mds.a.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5645753dea95] 2023-10-18T17:58:23.105 INFO:tasks.ceph.mds.a.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5645753a9b10] 2023-10-18T17:58:23.106 INFO:tasks.ceph.mds.a.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5645753adabf] 2023-10-18T17:58:23.106 INFO:tasks.ceph.mds.a.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5645753ae456] 2023-10-18T17:58:23.106 INFO:tasks.ceph.mds.a.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f96942bc710] 2023-10-18T17:58:23.106 INFO:tasks.ceph.mds.a.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f96942b9faf] 2023-10-18T17:58:23.106 INFO:tasks.ceph.mds.a.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9694388b21] 2023-10-18T17:58:23.107 INFO:tasks.ceph.mds.a.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9693daa609] 2023-10-18T17:58:23.107 INFO:tasks.ceph.mds.a.smithi072.stderr: 19: clone() 2023-10-18T17:58:23.107 INFO:tasks.ceph.mds.a.smithi072.stderr:2023-10-18T17:58:23.098+0000 7f968eb60700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:58:23.108 INFO:tasks.ceph.mds.a.smithi072.stderr: in thread 7f968eb60700 thread_name:ms_dispatch 2023-10-18T17:58:23.108 INFO:tasks.ceph.mds.a.smithi072.stderr: 2023-10-18T17:58:23.108 INFO:tasks.ceph.mds.a.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:58:23.108 INFO:tasks.ceph.mds.a.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f9693db6420] 2023-10-18T17:58:23.109 INFO:tasks.ceph.mds.a.smithi072.stderr: 2: gsignal() 2023-10-18T17:58:23.109 INFO:tasks.ceph.mds.a.smithi072.stderr: 3: abort() 2023-10-18T17:58:23.109 INFO:tasks.ceph.mds.a.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f9693c5e911] 2023-10-18T17:58:23.109 INFO:tasks.ceph.mds.a.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f9693c6a38c] 2023-10-18T17:58:23.109 INFO:tasks.ceph.mds.a.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f9693c6a3f7] 2023-10-18T17:58:23.110 INFO:tasks.ceph.mds.a.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f9693c6a6a9] 2023-10-18T17:58:23.110 INFO:tasks.ceph.mds.a.smithi072.stderr: 8: ceph-mds(+0x185107) [0x56457536b107] 2023-10-18T17:58:23.110 INFO:tasks.ceph.mds.a.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x56457536b72d] 2023-10-18T17:58:23.110 INFO:tasks.ceph.mds.a.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5645753dd09f] 2023-10-18T17:58:23.110 INFO:tasks.ceph.mds.a.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5645753dea95] 2023-10-18T17:58:23.111 INFO:tasks.ceph.mds.a.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5645753a9b10] 2023-10-18T17:58:23.111 INFO:tasks.ceph.mds.a.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5645753adabf] 2023-10-18T17:58:23.111 INFO:tasks.ceph.mds.a.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5645753ae456] 2023-10-18T17:58:23.111 INFO:tasks.ceph.mds.a.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f96942bc710] 2023-10-18T17:58:23.111 INFO:tasks.ceph.mds.a.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f96942b9faf] 2023-10-18T17:58:23.112 INFO:tasks.ceph.mds.a.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9694388b21] 2023-10-18T17:58:23.112 INFO:tasks.ceph.mds.a.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9693daa609] 2023-10-18T17:58:23.112 INFO:tasks.ceph.mds.a.smithi072.stderr: 19: clone() 2023-10-18T17:58:23.112 INFO:tasks.ceph.mds.a.smithi072.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:58:23.112 INFO:tasks.ceph.mds.a.smithi072.stderr: 2023-10-18T17:58:23.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.102+0000 7ff604ff9700 1 -- 172.21.15.72:0/3922804067 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff5f0022070 con 0x7ff6080bfd50 2023-10-18T17:58:23.113 INFO:tasks.ceph.mds.a.smithi072.stderr: 0> 2023-10-18T17:58:23.098+0000 7f968eb60700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:58:23.113 INFO:tasks.ceph.mds.a.smithi072.stderr: in thread 7f968eb60700 thread_name:ms_dispatch 2023-10-18T17:58:23.114 INFO:tasks.ceph.mds.a.smithi072.stderr: 2023-10-18T17:58:23.114 INFO:tasks.ceph.mds.a.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:58:23.114 INFO:tasks.ceph.mds.a.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f9693db6420] 2023-10-18T17:58:23.114 INFO:tasks.ceph.mds.a.smithi072.stderr: 2: gsignal() 2023-10-18T17:58:23.115 INFO:tasks.ceph.mds.a.smithi072.stderr: 3: abort() 2023-10-18T17:58:23.115 INFO:tasks.ceph.mds.a.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f9693c5e911] 2023-10-18T17:58:23.115 INFO:tasks.ceph.mds.a.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f9693c6a38c] 2023-10-18T17:58:23.115 INFO:tasks.ceph.mds.a.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f9693c6a3f7] 2023-10-18T17:58:23.115 INFO:tasks.ceph.mds.a.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f9693c6a6a9] 2023-10-18T17:58:23.116 INFO:tasks.ceph.mds.a.smithi072.stderr: 8: ceph-mds(+0x185107) [0x56457536b107] 2023-10-18T17:58:23.116 INFO:tasks.ceph.mds.a.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x56457536b72d] 2023-10-18T17:58:23.116 INFO:tasks.ceph.mds.a.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5645753dd09f] 2023-10-18T17:58:23.116 INFO:tasks.ceph.mds.a.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5645753dea95] 2023-10-18T17:58:23.116 INFO:tasks.ceph.mds.a.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5645753a9b10] 2023-10-18T17:58:23.116 INFO:tasks.ceph.mds.a.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5645753adabf] 2023-10-18T17:58:23.117 INFO:tasks.ceph.mds.a.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5645753ae456] 2023-10-18T17:58:23.117 INFO:tasks.ceph.mds.a.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f96942bc710] 2023-10-18T17:58:23.117 INFO:tasks.ceph.mds.a.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f96942b9faf] 2023-10-18T17:58:23.117 INFO:tasks.ceph.mds.a.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9694388b21] 2023-10-18T17:58:23.118 INFO:tasks.ceph.mds.a.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9693daa609] 2023-10-18T17:58:23.118 INFO:tasks.ceph.mds.a.smithi072.stderr: 19: clone() 2023-10-18T17:58:23.118 INFO:tasks.ceph.mds.a.smithi072.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:58:23.118 INFO:tasks.ceph.mds.a.smithi072.stderr: 2023-10-18T17:58:23.119 INFO:tasks.ceph.mds.a.smithi072.stderr: 0> 2023-10-18T17:58:23.098+0000 7f968eb60700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:58:23.119 INFO:tasks.ceph.mds.a.smithi072.stderr: in thread 7f968eb60700 thread_name:ms_dispatch 2023-10-18T17:58:23.119 INFO:tasks.ceph.mds.a.smithi072.stderr: 2023-10-18T17:58:23.119 INFO:tasks.ceph.mds.a.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:58:23.119 INFO:tasks.ceph.mds.a.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f9693db6420] 2023-10-18T17:58:23.120 INFO:tasks.ceph.mds.a.smithi072.stderr: 2: gsignal() 2023-10-18T17:58:23.120 INFO:tasks.ceph.mds.a.smithi072.stderr: 3: abort() 2023-10-18T17:58:23.120 INFO:tasks.ceph.mds.a.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f9693c5e911] 2023-10-18T17:58:23.120 INFO:tasks.ceph.mds.a.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f9693c6a38c] 2023-10-18T17:58:23.120 INFO:tasks.ceph.mds.a.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f9693c6a3f7] 2023-10-18T17:58:23.121 INFO:tasks.ceph.mds.a.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f9693c6a6a9] 2023-10-18T17:58:23.121 INFO:tasks.ceph.mds.a.smithi072.stderr: 8: ceph-mds(+0x185107) [0x56457536b107] 2023-10-18T17:58:23.121 INFO:tasks.ceph.mds.a.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x56457536b72d] 2023-10-18T17:58:23.121 INFO:tasks.ceph.mds.a.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5645753dd09f] 2023-10-18T17:58:23.121 INFO:tasks.ceph.mds.a.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5645753dea95] 2023-10-18T17:58:23.122 INFO:tasks.ceph.mds.a.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5645753a9b10] 2023-10-18T17:58:23.122 INFO:tasks.ceph.mds.a.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5645753adabf] 2023-10-18T17:58:23.122 INFO:tasks.ceph.mds.a.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5645753ae456] 2023-10-18T17:58:23.122 INFO:tasks.ceph.mds.a.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f96942bc710] 2023-10-18T17:58:23.122 INFO:tasks.ceph.mds.a.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f96942b9faf] 2023-10-18T17:58:23.123 INFO:tasks.ceph.mds.a.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9694388b21] 2023-10-18T17:58:23.123 INFO:tasks.ceph.mds.a.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f9693daa609] 2023-10-18T17:58:23.123 INFO:tasks.ceph.mds.a.smithi072.stderr: 19: clone() 2023-10-18T17:58:23.123 INFO:tasks.ceph.mds.a.smithi072.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:58:23.123 INFO:tasks.ceph.mds.a.smithi072.stderr: 2023-10-18T17:58:23.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff5e805a540 msgr2=0x7ff5e805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff5e805a540 0x7ff5e805ca00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff5f8009b90 tx=0x7ff5f8009060).stop 2023-10-18T17:58:23.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6080bfd50 msgr2=0x7ff6081520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6080bfd50 0x7ff6081520e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7ff6080c1e10 tx=0x7ff5f0016000).stop 2023-10-18T17:58:23.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 shutdown_connections 2023-10-18T17:58:23.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff5e805a540 0x7ff5e805ca00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff6080c4290 0x7ff608163df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6080bfd50 0x7ff6081520e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 --2- 172.21.15.72:0/3922804067 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff6080684e0 0x7ff608160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 >> 172.21.15.72:0/3922804067 conn(0x7ff6080667b0 msgr2=0x7ff608150a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:23.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 shutdown_connections 2023-10-18T17:58:23.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.110+0000 7ff60e69b700 1 -- 172.21.15.72:0/3922804067 wait complete. 2023-10-18T17:58:23.127 INFO:teuthology.orchestra.run.smithi072.stderr:new fs with metadata pool 3 and data pool 4 2023-10-18T17:58:23.144 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.294+0000 7f134d2f1700 1 Processor -- start 2023-10-18T17:58:23.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.294+0000 7f134d2f1700 1 -- start start 2023-10-18T17:58:23.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.294+0000 7f134d2f1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1348067d90 0x7f1348065b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.294+0000 7f134d2f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348153290 0x7f134815a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.294+0000 7f134d2f1700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13480681f0 con 0x7f13480b9b20 2023-10-18T17:58:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.294+0000 7f134d2f1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13480c0cf0 con 0x7f1348067d90 2023-10-18T17:58:23.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.294+0000 7f134d2f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1348153720 con 0x7f1348153290 2023-10-18T17:58:23.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f13467fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1348067d90 0x7f1348065b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f13467fc700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f1348067d90 0x7f1348065b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51090/0 (socket says 172.21.15.72:51090) 2023-10-18T17:58:23.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f13467fc700 1 -- 172.21.15.72:0/2107395398 learned_addr learned my addr 172.21.15.72:0/2107395398 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:23.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f1346ffd700 1 --1- 172.21.15.72:0/2107395398 >> v1:172.21.15.179:6789/0 conn(0x7f13480b9b20 0x7f13480645a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:60698/0 (socket says 172.21.15.72:60698) 2023-10-18T17:58:23.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f13477fe700 1 --2- 172.21.15.72:0/2107395398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348153290 0x7f134815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f13467fc700 1 -- 172.21.15.72:0/2107395398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348153290 msgr2=0x7f134815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:23.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f13467fc700 1 --2- 172.21.15.72:0/2107395398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348153290 0x7f134815a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f13467fc700 1 -- 172.21.15.72:0/2107395398 >> v1:172.21.15.179:6789/0 conn(0x7f13480b9b20 legacy=0x7f13480645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f13467fc700 1 -- 172.21.15.72:0/2107395398 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13480c2b70 con 0x7f1348067d90 2023-10-18T17:58:23.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f13467fc700 1 --2- 172.21.15.72:0/2107395398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1348067d90 0x7f1348065b20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f133c006fd0 tx=0x7f133c0090f0).ready entity=mon.1 client_cookie=7394f2f82164e27d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f1345ffb700 1 -- 172.21.15.72:0/2107395398 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c00b580 con 0x7f1348067d90 2023-10-18T17:58:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f1345ffb700 1 -- 172.21.15.72:0/2107395398 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c00b6e0 con 0x7f1348067d90 2023-10-18T17:58:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f1345ffb700 1 -- 172.21.15.72:0/2107395398 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c00ba60 con 0x7f1348067d90 2023-10-18T17:58:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f134d2f1700 1 -- 172.21.15.72:0/2107395398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1348067d90 msgr2=0x7f1348065b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f134d2f1700 1 --2- 172.21.15.72:0/2107395398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1348067d90 0x7f1348065b20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f133c006fd0 tx=0x7f133c0090f0).stop 2023-10-18T17:58:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f134d2f1700 1 -- 172.21.15.72:0/2107395398 shutdown_connections 2023-10-18T17:58:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f134d2f1700 1 --2- 172.21.15.72:0/2107395398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348153290 0x7f134815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f134d2f1700 1 --2- 172.21.15.72:0/2107395398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1348067d90 0x7f1348065b20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f134d2f1700 1 -- 172.21.15.72:0/2107395398 >> 172.21.15.72:0/2107395398 conn(0x7f1348069c80 msgr2=0x7f134806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f134d2f1700 1 -- 172.21.15.72:0/2107395398 shutdown_connections 2023-10-18T17:58:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.298+0000 7f134d2f1700 1 -- 172.21.15.72:0/2107395398 wait complete. 2023-10-18T17:58:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.302+0000 7f134d2f1700 1 Processor -- start 2023-10-18T17:58:23.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.302+0000 7f134d2f1700 1 -- start start 2023-10-18T17:58:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f134d2f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348067d90 0x7f1348160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f134d2f1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13480b9b20 0x7f134814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f134d2f1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1348153290 0x7f1348152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f134d2f1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1348160f70 con 0x7f1348153290 2023-10-18T17:58:23.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f134d2f1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480bbdc0 con 0x7f13480b9b20 2023-10-18T17:58:23.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f134d2f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13480bc060 con 0x7f1348067d90 2023-10-18T17:58:23.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13467fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13480b9b20 0x7f134814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f1346ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348067d90 0x7f1348160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13477fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1348153290 0x7f1348152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13467fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13480b9b20 0x7f134814ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51102/0 (socket says 172.21.15.72:51102) 2023-10-18T17:58:23.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13467fc700 1 -- 172.21.15.72:0/385310167 learned_addr learned my addr 172.21.15.72:0/385310167 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:23.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13477fe700 1 -- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348067d90 msgr2=0x7f1348160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13477fe700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348067d90 0x7f1348160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13477fe700 1 -- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13480b9b20 msgr2=0x7f134814ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13477fe700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13480b9b20 0x7f134814ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13477fe700 1 -- 172.21.15.72:0/385310167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c009370 con 0x7f1348153290 2023-10-18T17:58:23.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f13477fe700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1348153290 0x7f1348152310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f134000ecc0 tx=0x7f134000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:23.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f132effd700 1 -- 172.21.15.72:0/385310167 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134000c950 con 0x7f1348153290 2023-10-18T17:58:23.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f132effd700 1 -- 172.21.15.72:0/385310167 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134000f040 con 0x7f1348153290 2023-10-18T17:58:23.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13480bc340 con 0x7f1348153290 2023-10-18T17:58:23.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f132effd700 1 -- 172.21.15.72:0/385310167 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134000ccc0 con 0x7f1348153290 2023-10-18T17:58:23.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13480bc900 con 0x7f1348153290 2023-10-18T17:58:23.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.306+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13480baad0 con 0x7f1348153290 2023-10-18T17:58:23.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.310+0000 7f132effd700 1 -- 172.21.15.72:0/385310167 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1340013610 con 0x7f1348153290 2023-10-18T17:58:23.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.310+0000 7f132effd700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f133405a4e0 0x7f133405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.315 INFO:tasks.ceph.mds.a.smithi072.stderr:daemon-helper: command crashed with signal 6 2023-10-18T17:58:23.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.310+0000 7f132effd700 1 -- 172.21.15.72:0/385310167 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1340074fa0 con 0x7f1348153290 2023-10-18T17:58:23.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.310+0000 7f1346ffd700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f133405a4e0 0x7f133405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.314+0000 7f1346ffd700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f133405a4e0 0x7f133405c9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f1338007fd0 tx=0x7f1338007d30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:23.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.322+0000 7f132effd700 1 -- 172.21.15.72:0/385310167 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f13480baad0 con 0x7f1348153290 2023-10-18T17:58:23.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.486+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1348046f00 con 0x7f1348153290 2023-10-18T17:58:23.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.490+0000 7f132effd700 1 -- 172.21.15.72:0/385310167 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f13400486c0 con 0x7f1348153290 2023-10-18T17:58:23.494 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:23.495 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":21,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:23.090706+0000","last_up_change":"2023-10-18T17:58:14.550597+0000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T17:58:13.627096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T17:58:15.732786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T17:58:21.178314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T17:58:22.112742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":33232},{"type":"v1","addr":"172.21.15.72:6810","nonce":33232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6813","nonce":33232},{"type":"v1","addr":"172.21.15.72:6815","nonce":33232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":33232},{"type":"v1","addr":"172.21.15.72:6826","nonce":33232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":33232},{"type":"v1","addr":"172.21.15.72:6820","nonce":33232}]},"public_addr":"172.21.15.72:6810/33232","cluster_addr":"172.21.15.72:6815/33232","heartbeat_back_addr":"172.21.15.72:6826/33232","heartbeat_front_addr":"172.21.15.72:6820/33232","state":["exists","up"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6809","nonce":33229},{"type":"v1","addr":"172.21.15.72:6812","nonce":33229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":33229},{"type":"v1","addr":"172.21.15.72:6819","nonce":33229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":33229},{"type":"v1","addr":"172.21.15.72:6831","nonce":33229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":33229},{"type":"v1","addr":"172.21.15.72:6825","nonce":33229}]},"public_addr":"172.21.15.72:6812/33229","cluster_addr":"172.21.15.72:6819/33229","heartbeat_back_addr":"172.21.15.72:6831/33229","heartbeat_front_addr":"172.21.15.72:6825/33229","state":["exists","up"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":33231},{"type":"v1","addr":"172.21.15.72:6801","nonce":33231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":33231},{"type":"v1","addr":"172.21.15.72:6803","nonce":33231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":33231},{"type":"v1","addr":"172.21.15.72:6807","nonce":33231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":33231},{"type":"v1","addr":"172.21.15.72:6805","nonce":33231}]},"public_addr":"172.21.15.72:6801/33231","cluster_addr":"172.21.15.72:6803/33231","heartbeat_back_addr":"172.21.15.72:6807/33231","heartbeat_front_addr":"172.21.15.72:6805/33231","state":["exists","up"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":33228},{"type":"v1","addr":"172.21.15.72:6814","nonce":33228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":33228},{"type":"v1","addr":"172.21.15.72:6821","nonce":33228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":33228},{"type":"v1","addr":"172.21.15.72:6830","nonce":33228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":33228},{"type":"v1","addr":"172.21.15.72:6827","nonce":33228}]},"public_addr":"172.21.15.72:6814/33228","cluster_addr":"172.21.15.72:6821/33228","heartbeat_back_addr":"172.21.15.72:6830/33228","heartbeat_front_addr":"172.21.15.72:6827/33228","state":["exists","up"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":34034},{"type":"v1","addr":"172.21.15.179:6809","nonce":34034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":34034},{"type":"v1","addr":"172.21.15.179:6813","nonce":34034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":34034},{"type":"v1","addr":"172.21.15.179:6821","nonce":34034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":34034},{"type":"v1","addr":"172.21.15.179:6817","nonce":34034}]},"public_addr":"172.21.15.179:6809/34034","cluster_addr":"172.21.15.179:6813/34034","heartbeat_back_addr":"172.21.15.179:6821/34034","heartbeat_front_addr":"172.21.15.179:6817/34034","state":["exists","up"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":34044},{"type":"v1","addr":"172.21.15.179:6801","nonce":34044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":34044},{"type":"v1","addr":"172.21.15.179:6803","nonce":34044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":34044},{"type":"v1","addr":"172.21.15.179:6807","nonce":34044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":34044},{"type":"v1","addr":"172.21.15.179:6805","nonce":34044}]},"public_addr":"172.21.15.179:6801/34044","cluster_addr":"172.21.15.179:6803/34044","heartbeat_back_addr":"172.21.15.179:6807/34044","heartbeat_front_addr":"172.21.15.179:6805/34044","state":["exists","up"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":34036},{"type":"v1","addr":"172.21.15.179:6825","nonce":34036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":34036},{"type":"v1","addr":"172.21.15.179:6827","nonce":34036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":34036},{"type":"v1","addr":"172.21.15.179:6831","nonce":34036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":34036},{"type":"v1","addr":"172.21.15.179:6829","nonce":34036}]},"public_addr":"172.21.15.179:6825/34036","cluster_addr":"172.21.15.179:6827/34036","heartbeat_back_addr":"172.21.15.179:6831/34036","heartbeat_front_addr":"172.21.15.179:6829/34036","state":["exists","up"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":34035},{"type":"v1","addr":"172.21.15.179:6812","nonce":34035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":34035},{"type":"v1","addr":"172.21.15.179:6816","nonce":34035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":34035},{"type":"v1","addr":"172.21.15.179:6823","nonce":34035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":34035},{"type":"v1","addr":"172.21.15.179:6820","nonce":34035}]},"public_addr":"172.21.15.179:6812/34035","cluster_addr":"172.21.15.179:6816/34035","heartbeat_back_addr":"172.21.15.179:6823/34035","heartbeat_front_addr":"172.21.15.179:6820/34035","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.697876+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.661119+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.579136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.792036+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.069244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:11.971339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.176699+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.197004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:23.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f133405a4e0 msgr2=0x7f133405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f133405a4e0 0x7f133405c9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f1338007fd0 tx=0x7f1338007d30).stop 2023-10-18T17:58:23.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1348153290 msgr2=0x7f1348152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1348153290 0x7f1348152310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f134000ecc0 tx=0x7f134000b040).stop 2023-10-18T17:58:23.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 shutdown_connections 2023-10-18T17:58:23.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f133405a4e0 0x7f133405c9a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1348153290 0x7f1348152310 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13480b9b20 0x7f134814ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 --2- 172.21.15.72:0/385310167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1348067d90 0x7f1348160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 >> 172.21.15.72:0/385310167 conn(0x7f1348069c80 msgr2=0x7f13480645a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:23.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 shutdown_connections 2023-10-18T17:58:23.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.494+0000 7f134d2f1700 1 -- 172.21.15.72:0/385310167 wait complete. 2023-10-18T17:58:23.519 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:23.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.674+0000 7f51a2c39700 1 Processor -- start 2023-10-18T17:58:23.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.674+0000 7f51a2c39700 1 -- start start 2023-10-18T17:58:23.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a2c39700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f519c062270 0x7f519c062650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a2c39700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c0c1610 con 0x7f519c066340 2023-10-18T17:58:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a2c39700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c0c2dd0 con 0x7f519c14d020 2023-10-18T17:58:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a2c39700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c153570 con 0x7f519c062270 2023-10-18T17:58:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a1c37700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f519c062270 0x7f519c062650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a1c37700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f519c062270 0x7f519c062650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58722/0 (socket says 172.21.15.72:58722) 2023-10-18T17:58:23.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a1c37700 1 -- 172.21.15.72:0/3546535616 learned_addr learned my addr 172.21.15.72:0/3546535616 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:23.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a1436700 1 --1- 172.21.15.72:0/3546535616 >> v1:172.21.15.72:6789/0 conn(0x7f519c066340 0x7f519c0c0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50512/0 (socket says 172.21.15.72:50512) 2023-10-18T17:58:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a0c35700 1 -- 172.21.15.72:0/3546535616 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142938747 0 0) 0x7f519c0c2dd0 con 0x7f519c14d020 2023-10-18T17:58:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a1c37700 1 -- 172.21.15.72:0/3546535616 >> v1:172.21.15.179:6789/0 conn(0x7f519c14d020 legacy=0x7f519c14d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a1c37700 1 -- 172.21.15.72:0/3546535616 >> v1:172.21.15.72:6789/0 conn(0x7f519c066340 legacy=0x7f519c0c0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a1c37700 1 -- 172.21.15.72:0/3546535616 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c153850 con 0x7f519c062270 2023-10-18T17:58:23.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a1c37700 1 --2- 172.21.15.72:0/3546535616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c062270 0x7f519c062650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f5198006fd0 tx=0x7f51980090f0).ready entity=mon.0 client_cookie=5f6caba5b140f877 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:23.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a0c35700 1 -- 172.21.15.72:0/3546535616 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800f510 con 0x7f519c062270 2023-10-18T17:58:23.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a0c35700 1 -- 172.21.15.72:0/3546535616 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519800f670 con 0x7f519c062270 2023-10-18T17:58:23.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.678+0000 7f51a0c35700 1 -- 172.21.15.72:0/3546535616 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800f9f0 con 0x7f519c062270 2023-10-18T17:58:23.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.682+0000 7f51a2c39700 1 -- 172.21.15.72:0/3546535616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c062270 msgr2=0x7f519c062650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.682+0000 7f51a2c39700 1 --2- 172.21.15.72:0/3546535616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c062270 0x7f519c062650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f5198006fd0 tx=0x7f51980090f0).stop 2023-10-18T17:58:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.682+0000 7f51a2c39700 1 -- 172.21.15.72:0/3546535616 shutdown_connections 2023-10-18T17:58:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.682+0000 7f51a2c39700 1 --2- 172.21.15.72:0/3546535616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c062270 0x7f519c062650 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.682+0000 7f51a2c39700 1 -- 172.21.15.72:0/3546535616 >> 172.21.15.72:0/3546535616 conn(0x7f519c05ddc0 msgr2=0x7f519c05e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:23.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.682+0000 7f51a2c39700 1 -- 172.21.15.72:0/3546535616 shutdown_connections 2023-10-18T17:58:23.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.682+0000 7f51a2c39700 1 -- 172.21.15.72:0/3546535616 wait complete. 2023-10-18T17:58:23.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.682+0000 7f51a2c39700 1 Processor -- start 2023-10-18T17:58:23.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2c39700 1 -- start start 2023-10-18T17:58:23.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2c39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c062270 0x7f519c0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2c39700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f519c066340 0x7f519c0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2c39700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c14d020 0x7f519c1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2c39700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c15ff20 con 0x7f519c14d020 2023-10-18T17:58:23.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2c39700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0c6c50 con 0x7f519c062270 2023-10-18T17:58:23.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2c39700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c1601e0 con 0x7f519c066340 2023-10-18T17:58:23.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2438700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c14d020 0x7f519c1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2438700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c14d020 0x7f519c1525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58732/0 (socket says 172.21.15.72:58732) 2023-10-18T17:58:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2438700 1 -- 172.21.15.72:0/1153251708 learned_addr learned my addr 172.21.15.72:0/1153251708 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a1c37700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c062270 0x7f519c0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a1436700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f519c066340 0x7f519c0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2438700 1 -- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f519c066340 msgr2=0x7f519c0c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2438700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f519c066340 0x7f519c0c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2438700 1 -- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c062270 msgr2=0x7f519c0c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2438700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c062270 0x7f519c0c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2438700 1 -- 172.21.15.72:0/1153251708 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198009370 con 0x7f519c14d020 2023-10-18T17:58:23.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a1436700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f519c066340 0x7f519c0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-18T17:58:23.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2438700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c14d020 0x7f519c1525b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f519400c810 tx=0x7f5194001500).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:23.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f5191ffb700 1 -- 172.21.15.72:0/1153251708 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194010510 con 0x7f519c14d020 2023-10-18T17:58:23.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f5191ffb700 1 -- 172.21.15.72:0/1153251708 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5194010670 con 0x7f519c14d020 2023-10-18T17:58:23.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c1604c0 con 0x7f519c14d020 2023-10-18T17:58:23.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f5191ffb700 1 -- 172.21.15.72:0/1153251708 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194010a10 con 0x7f519c14d020 2023-10-18T17:58:23.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.686+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c160a80 con 0x7f519c14d020 2023-10-18T17:58:23.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.690+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c1634f0 con 0x7f519c14d020 2023-10-18T17:58:23.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.690+0000 7f5191ffb700 1 -- 172.21.15.72:0/1153251708 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5194012440 con 0x7f519c14d020 2023-10-18T17:58:23.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.690+0000 7f5191ffb700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f518005a540 0x7f518005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:23.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.690+0000 7f5191ffb700 1 -- 172.21.15.72:0/1153251708 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f51940741d0 con 0x7f519c14d020 2023-10-18T17:58:23.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.690+0000 7f51a1c37700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f518005a540 0x7f518005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:23.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.690+0000 7f51a1c37700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f518005a540 0x7f518005ca00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5198009130 tx=0x7f5198017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:23.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.698+0000 7f5191ffb700 1 -- 172.21.15.72:0/1153251708 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f519c1634f0 con 0x7f519c14d020 2023-10-18T17:58:23.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.866+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f519c1634f0 con 0x7f519c14d020 2023-10-18T17:58:23.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.870+0000 7f5191ffb700 1 -- 172.21.15.72:0/1153251708 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f519c1634f0 con 0x7f519c14d020 2023-10-18T17:58:23.874 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:23.874 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":21,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:23.090706+0000","last_up_change":"2023-10-18T17:58:14.550597+0000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T17:58:13.627096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T17:58:15.732786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T17:58:21.178314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T17:58:22.112742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":33232},{"type":"v1","addr":"172.21.15.72:6810","nonce":33232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6813","nonce":33232},{"type":"v1","addr":"172.21.15.72:6815","nonce":33232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":33232},{"type":"v1","addr":"172.21.15.72:6826","nonce":33232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":33232},{"type":"v1","addr":"172.21.15.72:6820","nonce":33232}]},"public_addr":"172.21.15.72:6810/33232","cluster_addr":"172.21.15.72:6815/33232","heartbeat_back_addr":"172.21.15.72:6826/33232","heartbeat_front_addr":"172.21.15.72:6820/33232","state":["exists","up"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6809","nonce":33229},{"type":"v1","addr":"172.21.15.72:6812","nonce":33229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":33229},{"type":"v1","addr":"172.21.15.72:6819","nonce":33229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":33229},{"type":"v1","addr":"172.21.15.72:6831","nonce":33229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":33229},{"type":"v1","addr":"172.21.15.72:6825","nonce":33229}]},"public_addr":"172.21.15.72:6812/33229","cluster_addr":"172.21.15.72:6819/33229","heartbeat_back_addr":"172.21.15.72:6831/33229","heartbeat_front_addr":"172.21.15.72:6825/33229","state":["exists","up"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":33231},{"type":"v1","addr":"172.21.15.72:6801","nonce":33231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":33231},{"type":"v1","addr":"172.21.15.72:6803","nonce":33231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":33231},{"type":"v1","addr":"172.21.15.72:6807","nonce":33231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":33231},{"type":"v1","addr":"172.21.15.72:6805","nonce":33231}]},"public_addr":"172.21.15.72:6801/33231","cluster_addr":"172.21.15.72:6803/33231","heartbeat_back_addr":"172.21.15.72:6807/33231","heartbeat_front_addr":"172.21.15.72:6805/33231","state":["exists","up"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":33228},{"type":"v1","addr":"172.21.15.72:6814","nonce":33228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":33228},{"type":"v1","addr":"172.21.15.72:6821","nonce":33228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":33228},{"type":"v1","addr":"172.21.15.72:6830","nonce":33228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":33228},{"type":"v1","addr":"172.21.15.72:6827","nonce":33228}]},"public_addr":"172.21.15.72:6814/33228","cluster_addr":"172.21.15.72:6821/33228","heartbeat_back_addr":"172.21.15.72:6830/33228","heartbeat_front_addr":"172.21.15.72:6827/33228","state":["exists","up"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":34034},{"type":"v1","addr":"172.21.15.179:6809","nonce":34034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":34034},{"type":"v1","addr":"172.21.15.179:6813","nonce":34034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":34034},{"type":"v1","addr":"172.21.15.179:6821","nonce":34034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":34034},{"type":"v1","addr":"172.21.15.179:6817","nonce":34034}]},"public_addr":"172.21.15.179:6809/34034","cluster_addr":"172.21.15.179:6813/34034","heartbeat_back_addr":"172.21.15.179:6821/34034","heartbeat_front_addr":"172.21.15.179:6817/34034","state":["exists","up"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":34044},{"type":"v1","addr":"172.21.15.179:6801","nonce":34044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":34044},{"type":"v1","addr":"172.21.15.179:6803","nonce":34044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":34044},{"type":"v1","addr":"172.21.15.179:6807","nonce":34044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":34044},{"type":"v1","addr":"172.21.15.179:6805","nonce":34044}]},"public_addr":"172.21.15.179:6801/34044","cluster_addr":"172.21.15.179:6803/34044","heartbeat_back_addr":"172.21.15.179:6807/34044","heartbeat_front_addr":"172.21.15.179:6805/34044","state":["exists","up"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":34036},{"type":"v1","addr":"172.21.15.179:6825","nonce":34036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":34036},{"type":"v1","addr":"172.21.15.179:6827","nonce":34036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":34036},{"type":"v1","addr":"172.21.15.179:6831","nonce":34036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":34036},{"type":"v1","addr":"172.21.15.179:6829","nonce":34036}]},"public_addr":"172.21.15.179:6825/34036","cluster_addr":"172.21.15.179:6827/34036","heartbeat_back_addr":"172.21.15.179:6831/34036","heartbeat_front_addr":"172.21.15.179:6829/34036","state":["exists","up"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":34035},{"type":"v1","addr":"172.21.15.179:6812","nonce":34035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":34035},{"type":"v1","addr":"172.21.15.179:6816","nonce":34035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":34035},{"type":"v1","addr":"172.21.15.179:6823","nonce":34035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":34035},{"type":"v1","addr":"172.21.15.179:6820","nonce":34035}]},"public_addr":"172.21.15.179:6812/34035","cluster_addr":"172.21.15.179:6816/34035","heartbeat_back_addr":"172.21.15.179:6823/34035","heartbeat_front_addr":"172.21.15.179:6820/34035","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.697876+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.661119+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.579136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.792036+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.069244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:11.971339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.176699+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.197004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:23.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f518005a540 msgr2=0x7f518005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f518005a540 0x7f518005ca00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5198009130 tx=0x7f5198017000).stop 2023-10-18T17:58:23.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c14d020 msgr2=0x7f519c1525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:23.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c14d020 0x7f519c1525b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f519400c810 tx=0x7f5194001500).stop 2023-10-18T17:58:23.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 shutdown_connections 2023-10-18T17:58:23.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f518005a540 0x7f518005ca00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f519c14d020 0x7f519c1525b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f519c066340 0x7f519c0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 --2- 172.21.15.72:0/1153251708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f519c062270 0x7f519c0c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:23.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 >> 172.21.15.72:0/1153251708 conn(0x7f519c05ddc0 msgr2=0x7f519c14aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:23.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 shutdown_connections 2023-10-18T17:58:23.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:23.874+0000 7f51a2c39700 1 -- 172.21.15.72:0/1153251708 wait complete. 2023-10-18T17:58:23.898 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T17:58:24.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.054+0000 7f417cc8f700 1 Processor -- start 2023-10-18T17:58:24.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.054+0000 7f417cc8f700 1 -- start start 2023-10-18T17:58:24.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.058+0000 7f417cc8f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4178067d70 0x7f4178069580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:24.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.058+0000 7f417cc8f700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417814ff90 con 0x7f417814d690 2023-10-18T17:58:24.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.058+0000 7f417cc8f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41780690f0 con 0x7f4178069a70 2023-10-18T17:58:24.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.058+0000 7f417cc8f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417806a4e0 con 0x7f4178067d70 2023-10-18T17:58:24.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.058+0000 7f416effd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4178067d70 0x7f4178069580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:24.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.058+0000 7f4177fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f4178069a70 0x7f4178157330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:60710/0 (socket says 172.21.15.72:60710) 2023-10-18T17:58:24.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.058+0000 7f4177fff700 1 -- 172.21.15.72:0/1765760362 learned_addr learned my addr 172.21.15.72:0/1765760362 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:24.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.058+0000 7f41777fe700 1 --1- 172.21.15.72:0/1765760362 >> v1:172.21.15.72:6789/0 conn(0x7f417814d690 0x7f41780be8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50522/0 (socket says 172.21.15.72:50522) 2023-10-18T17:58:24.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f4176ffd700 1 -- 172.21.15.72:0/1765760362 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089289044 0 0) 0x7f41780690f0 con 0x7f4178069a70 2023-10-18T17:58:24.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f4176ffd700 1 -- 172.21.15.72:0/1765760362 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41640033f0 con 0x7f4178069a70 2023-10-18T17:58:24.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f416effd700 1 -- 172.21.15.72:0/1765760362 >> v1:172.21.15.179:6789/0 conn(0x7f4178069a70 legacy=0x7f4178157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:24.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f416effd700 1 -- 172.21.15.72:0/1765760362 >> v1:172.21.15.72:6789/0 conn(0x7f417814d690 legacy=0x7f41780be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:24.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f4176ffd700 1 -- 172.21.15.72:0/1765760362 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018338328 0 0) 0x7f417814ff90 con 0x7f417814d690 2023-10-18T17:58:24.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f416effd700 1 -- 172.21.15.72:0/1765760362 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41781501e0 con 0x7f4178067d70 2023-10-18T17:58:24.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f416effd700 1 --2- 172.21.15.72:0/1765760362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 0x7f4178069580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4160009760 tx=0x7f4160009060).ready entity=mon.1 client_cookie=7ccda831d085fcb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:24.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f4176ffd700 1 -- 172.21.15.72:0/1765760362 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416000b590 con 0x7f4178067d70 2023-10-18T17:58:24.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f4176ffd700 1 -- 172.21.15.72:0/1765760362 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416000b6f0 con 0x7f4178067d70 2023-10-18T17:58:24.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f4176ffd700 1 -- 172.21.15.72:0/1765760362 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416000ba70 con 0x7f4178067d70 2023-10-18T17:58:24.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f417cc8f700 1 -- 172.21.15.72:0/1765760362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 msgr2=0x7f4178069580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:24.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f417cc8f700 1 --2- 172.21.15.72:0/1765760362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 0x7f4178069580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f4160009760 tx=0x7f4160009060).stop 2023-10-18T17:58:24.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f417cc8f700 1 -- 172.21.15.72:0/1765760362 shutdown_connections 2023-10-18T17:58:24.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f417cc8f700 1 --2- 172.21.15.72:0/1765760362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 0x7f4178069580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:24.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f417cc8f700 1 -- 172.21.15.72:0/1765760362 >> 172.21.15.72:0/1765760362 conn(0x7f41780645a0 msgr2=0x7f41780649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:24.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f417cc8f700 1 -- 172.21.15.72:0/1765760362 shutdown_connections 2023-10-18T17:58:24.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.062+0000 7f417cc8f700 1 -- 172.21.15.72:0/1765760362 wait complete. 2023-10-18T17:58:24.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f417cc8f700 1 Processor -- start 2023-10-18T17:58:24.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f417cc8f700 1 -- start start 2023-10-18T17:58:24.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f417cc8f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 0x7f417814b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:24.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f417cc8f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178069a70 0x7f4178160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:24.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f417cc8f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f417814d690 0x7f417814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:24.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f417cc8f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417814ecb0 con 0x7f4178069a70 2023-10-18T17:58:24.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f417cc8f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4178160f30 con 0x7f4178067d70 2023-10-18T17:58:24.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f417cc8f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417814f630 con 0x7f417814d690 2023-10-18T17:58:24.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178069a70 0x7f4178160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:24.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178069a70 0x7f4178160a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58740/0 (socket says 172.21.15.72:58740) 2023-10-18T17:58:24.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416effd700 1 -- 172.21.15.72:0/1534584457 learned_addr learned my addr 172.21.15.72:0/1534584457 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:24.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f4177fff700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f417814d690 0x7f417814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:24.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f41777fe700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 0x7f417814b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:24.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416effd700 1 -- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f417814d690 msgr2=0x7f417814f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:24.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416effd700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f417814d690 0x7f417814f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:24.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416effd700 1 -- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 msgr2=0x7f417814b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:24.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416effd700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 0x7f417814b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:24.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416effd700 1 -- 172.21.15.72:0/1534584457 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41600092a0 con 0x7f4178069a70 2023-10-18T17:58:24.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f41777fe700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 0x7f417814b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:24.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416effd700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178069a70 0x7f4178160a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f4160009e00 tx=0x7f4160017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:24.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.066+0000 7f416ffff700 1 -- 172.21.15.72:0/1534584457 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416000b860 con 0x7f4178069a70 2023-10-18T17:58:24.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.070+0000 7f416ffff700 1 -- 172.21.15.72:0/1534584457 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416000b9c0 con 0x7f4178069a70 2023-10-18T17:58:24.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.070+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417814ddc0 con 0x7f4178069a70 2023-10-18T17:58:24.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.070+0000 7f416ffff700 1 -- 172.21.15.72:0/1534584457 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416001b660 con 0x7f4178069a70 2023-10-18T17:58:24.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.070+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417814e380 con 0x7f4178069a70 2023-10-18T17:58:24.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.070+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4178046f00 con 0x7f4178069a70 2023-10-18T17:58:24.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.070+0000 7f416ffff700 1 -- 172.21.15.72:0/1534584457 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f416001b9d0 con 0x7f4178069a70 2023-10-18T17:58:24.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.074+0000 7f416ffff700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415805a510 0x7f415805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:24.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.074+0000 7f416ffff700 1 -- 172.21.15.72:0/1534584457 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4160076060 con 0x7f4178069a70 2023-10-18T17:58:24.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.074+0000 7f41777fe700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415805a510 0x7f415805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:24.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.074+0000 7f41777fe700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415805a510 0x7f415805c9d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f41680057e0 tx=0x7f4168007cb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:24.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.078+0000 7f416ffff700 1 -- 172.21.15.72:0/1534584457 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f4160049840 con 0x7f4178069a70 2023-10-18T17:58:24.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:24.282+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f41780a7ab0 con 0x7f4178069a70 2023-10-18T17:58:25.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.106+0000 7f416ffff700 1 -- 172.21.15.72:0/1534584457 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f416004ce60 con 0x7f4178069a70 2023-10-18T17:58:25.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415805a510 msgr2=0x7f415805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:25.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415805a510 0x7f415805c9d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f41680057e0 tx=0x7f4168007cb0).stop 2023-10-18T17:58:25.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178069a70 msgr2=0x7f4178160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:25.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178069a70 0x7f4178160a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f4160009e00 tx=0x7f4160017000).stop 2023-10-18T17:58:25.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 shutdown_connections 2023-10-18T17:58:25.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415805a510 0x7f415805c9d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:25.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f417814d690 0x7f417814f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:25.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4178069a70 0x7f4178160a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:25.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 --2- 172.21.15.72:0/1534584457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4178067d70 0x7f417814b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:25.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 >> 172.21.15.72:0/1534584457 conn(0x7f41780645a0 msgr2=0x7f4178066050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:25.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 shutdown_connections 2023-10-18T17:58:25.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.110+0000 7f417cc8f700 1 -- 172.21.15.72:0/1534584457 wait complete. 2023-10-18T17:58:25.136 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2023-10-18T17:58:25.137 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-10-18T17:58:25.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.298+0000 7f98ecee4700 1 Processor -- start 2023-10-18T17:58:25.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.298+0000 7f98ecee4700 1 -- start start 2023-10-18T17:58:25.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98ecee4700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f98e8067d70 0x7f98e8069580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:25.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98ecee4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98e8069a70 0x7f98e8157330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:25.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98ecee4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e814ff90 con 0x7f98e814d690 2023-10-18T17:58:25.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98ecee4700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e806a4e0 con 0x7f98e8067d70 2023-10-18T17:58:25.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98ecee4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e80690f0 con 0x7f98e8069a70 2023-10-18T17:58:25.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e77fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f98e814d690 0x7f98e80be8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:60718/0 (socket says 172.21.15.72:60718) 2023-10-18T17:58:25.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e77fe700 1 -- 172.21.15.72:0/3210570442 learned_addr learned my addr 172.21.15.72:0/3210570442 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:25.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e7fff700 1 --2- 172.21.15.72:0/3210570442 >> v2:172.21.15.179:3300/0 conn(0x7f98e8069a70 0x7f98e8157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:25.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e6ffd700 1 --2- 172.21.15.72:0/3210570442 >> v2:172.21.15.72:3300/0 conn(0x7f98e8067d70 0x7f98e8069580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:25.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e67fc700 1 -- 172.21.15.72:0/3210570442 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458478571 0 0) 0x7f98e814ff90 con 0x7f98e814d690 2023-10-18T17:58:25.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e67fc700 1 -- 172.21.15.72:0/3210570442 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98d00033f0 con 0x7f98e814d690 2023-10-18T17:58:25.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e6ffd700 1 -- 172.21.15.72:0/3210570442 >> v2:172.21.15.179:3300/0 conn(0x7f98e8069a70 msgr2=0x7f98e8157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:25.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e6ffd700 1 --2- 172.21.15.72:0/3210570442 >> v2:172.21.15.179:3300/0 conn(0x7f98e8069a70 0x7f98e8157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:25.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e6ffd700 1 -- 172.21.15.72:0/3210570442 >> v1:172.21.15.179:6789/0 conn(0x7f98e814d690 legacy=0x7f98e80be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:25.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e6ffd700 1 -- 172.21.15.72:0/3210570442 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e81501e0 con 0x7f98e8067d70 2023-10-18T17:58:25.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e7fff700 1 --2- 172.21.15.72:0/3210570442 >> v2:172.21.15.179:3300/0 conn(0x7f98e8069a70 0x7f98e8157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:25.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e6ffd700 1 --2- 172.21.15.72:0/3210570442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 0x7f98e8069580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f98e80bfdb0 tx=0x7f98dc009060).ready entity=mon.0 client_cookie=6e63d664ff9361ac server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:25.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e67fc700 1 -- 172.21.15.72:0/3210570442 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00f6b0 con 0x7f98e8067d70 2023-10-18T17:58:25.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e67fc700 1 -- 172.21.15.72:0/3210570442 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc00f810 con 0x7f98e8067d70 2023-10-18T17:58:25.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.302+0000 7f98e67fc700 1 -- 172.21.15.72:0/3210570442 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00fb90 con 0x7f98e8067d70 2023-10-18T17:58:25.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 -- 172.21.15.72:0/3210570442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 msgr2=0x7f98e8069580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:25.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 --2- 172.21.15.72:0/3210570442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 0x7f98e8069580 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f98e80bfdb0 tx=0x7f98dc009060).stop 2023-10-18T17:58:25.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 -- 172.21.15.72:0/3210570442 shutdown_connections 2023-10-18T17:58:25.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 --2- 172.21.15.72:0/3210570442 >> v2:172.21.15.179:3300/0 conn(0x7f98e8069a70 0x7f98e8157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:25.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 --2- 172.21.15.72:0/3210570442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 0x7f98e8069580 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:25.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 -- 172.21.15.72:0/3210570442 >> 172.21.15.72:0/3210570442 conn(0x7f98e80645a0 msgr2=0x7f98e80649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:25.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 -- 172.21.15.72:0/3210570442 shutdown_connections 2023-10-18T17:58:25.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 -- 172.21.15.72:0/3210570442 wait complete. 2023-10-18T17:58:25.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 Processor -- start 2023-10-18T17:58:25.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.306+0000 7f98ecee4700 1 -- start start 2023-10-18T17:58:25.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98ecee4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 0x7f98e814c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:25.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98ecee4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98e8069a70 0x7f98e80a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:25.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98ecee4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98e814d690 0x7f98e80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:25.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98ecee4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e815fe40 con 0x7f98e8067d70 2023-10-18T17:58:25.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98ecee4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e8151050 con 0x7f98e8069a70 2023-10-18T17:58:25.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98ecee4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e8151e50 con 0x7f98e814d690 2023-10-18T17:58:25.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e77fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 0x7f98e814c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:25.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e77fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 0x7f98e814c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58758/0 (socket says 172.21.15.72:58758) 2023-10-18T17:58:25.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e77fe700 1 -- 172.21.15.72:0/369660137 learned_addr learned my addr 172.21.15.72:0/369660137 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:25.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e7fff700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98e814d690 0x7f98e80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:25.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e6ffd700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98e8069a70 0x7f98e80a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:25.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e77fe700 1 -- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98e814d690 msgr2=0x7f98e80a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:25.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e77fe700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98e814d690 0x7f98e80a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:25.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e77fe700 1 -- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98e8069a70 msgr2=0x7f98e80a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:25.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e77fe700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98e8069a70 0x7f98e80a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:25.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e77fe700 1 -- 172.21.15.72:0/369660137 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e0009a20 con 0x7f98e8067d70 2023-10-18T17:58:25.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98e77fe700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 0x7f98e814c700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f98d800c420 tx=0x7f98d8007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:25.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98cf7fe700 1 -- 172.21.15.72:0/369660137 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800ce20 con 0x7f98e8067d70 2023-10-18T17:58:25.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98cf7fe700 1 -- 172.21.15.72:0/369660137 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80072e0 con 0x7f98e8067d70 2023-10-18T17:58:25.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc0092a0 con 0x7f98e8067d70 2023-10-18T17:58:25.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98cf7fe700 1 -- 172.21.15.72:0/369660137 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8007660 con 0x7f98e8067d70 2023-10-18T17:58:25.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.310+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e8151800 con 0x7f98e8067d70 2023-10-18T17:58:25.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.314+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e8046f00 con 0x7f98e8067d70 2023-10-18T17:58:25.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.314+0000 7f98cf7fe700 1 -- 172.21.15.72:0/369660137 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f98d80059b0 con 0x7f98e8067d70 2023-10-18T17:58:25.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.314+0000 7f98cf7fe700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f98d405a510 0x7f98d405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:25.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.318+0000 7f98cf7fe700 1 -- 172.21.15.72:0/369660137 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f98d8073f60 con 0x7f98e8067d70 2023-10-18T17:58:25.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.318+0000 7f98e6ffd700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f98d405a510 0x7f98d405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:25.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.318+0000 7f98e6ffd700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f98d405a510 0x7f98d405c9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f98dc00ffd0 tx=0x7f98dc019000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:25.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.322+0000 7f98cf7fe700 1 -- 172.21.15.72:0/369660137 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f98d8047680 con 0x7f98e8067d70 2023-10-18T17:58:25.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:25.514+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f98e81504c0 con 0x7f98e8067d70 2023-10-18T17:58:26.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.114+0000 7f98cf7fe700 1 -- 172.21.15.72:0/369660137 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f98d804aca0 con 0x7f98e8067d70 2023-10-18T17:58:26.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f98d405a510 msgr2=0x7f98d405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:26.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f98d405a510 0x7f98d405c9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f98dc00ffd0 tx=0x7f98dc019000).stop 2023-10-18T17:58:26.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 msgr2=0x7f98e814c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:26.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 0x7f98e814c700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f98d800c420 tx=0x7f98d8007c80).stop 2023-10-18T17:58:26.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 shutdown_connections 2023-10-18T17:58:26.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f98d405a510 0x7f98d405c9d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:26.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98e814d690 0x7f98e80a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:26.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98e8069a70 0x7f98e80a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:26.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 --2- 172.21.15.72:0/369660137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98e8067d70 0x7f98e814c700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:26.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 >> 172.21.15.72:0/369660137 conn(0x7f98e80645a0 msgr2=0x7f98e814ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:26.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 shutdown_connections 2023-10-18T17:58:26.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.118+0000 7f98ecee4700 1 -- 172.21.15.72:0/369660137 wait complete. 2023-10-18T17:58:26.149 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T17:58:26.329 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:58:26.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.326+0000 7fcb1dc8f700 1 Processor -- start 2023-10-18T17:58:26.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.326+0000 7fcb1dc8f700 1 -- start start 2023-10-18T17:58:26.336 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:58:26.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb1dc8f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb180bec80 0x7fcb180bc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:26.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb1dc8f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fcb180bf060 0x7fcb18068710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:26.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb1dc8f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb180bd5e0 con 0x7fcb18068c00 2023-10-18T17:58:26.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb1dc8f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb180666a0 con 0x7fcb180bf060 2023-10-18T17:58:26.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb1dc8f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18066170 con 0x7fcb180bec80 2023-10-18T17:58:26.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb1d48e700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcb18068c00 0x7fcb18157720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:60722/0 (socket says 172.21.15.72:60722) 2023-10-18T17:58:26.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb1d48e700 1 -- 172.21.15.72:0/3798150066 learned_addr learned my addr 172.21.15.72:0/3798150066 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:26.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb1cc8d700 1 --2- 172.21.15.72:0/3798150066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb180bec80 0x7fcb180bc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:26.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb17fff700 1 --2- 172.21.15.72:0/3798150066 >> v2:172.21.15.179:3300/0 conn(0x7fcb180bf060 0x7fcb18068710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:26.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb177fe700 1 -- 172.21.15.72:0/3798150066 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724867509 0 0) 0x7fcb180bd5e0 con 0x7fcb18068c00 2023-10-18T17:58:26.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb177fe700 1 -- 172.21.15.72:0/3798150066 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb080033f0 con 0x7fcb18068c00 2023-10-18T17:58:26.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb17fff700 1 -- 172.21.15.72:0/3798150066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb180bec80 msgr2=0x7fcb180bc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:26.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb17fff700 1 --2- 172.21.15.72:0/3798150066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb180bec80 0x7fcb180bc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:26.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb17fff700 1 -- 172.21.15.72:0/3798150066 >> v1:172.21.15.179:6789/0 conn(0x7fcb18068c00 legacy=0x7fcb18157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:26.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb17fff700 1 -- 172.21.15.72:0/3798150066 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb18068280 con 0x7fcb180bf060 2023-10-18T17:58:26.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb17fff700 1 --2- 172.21.15.72:0/3798150066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bf060 0x7fcb18068710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fcb04009760 tx=0x7fcb04009060).ready entity=mon.1 client_cookie=a5711070ae1cd48 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:26.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb177fe700 1 -- 172.21.15.72:0/3798150066 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb040090d0 con 0x7fcb180bf060 2023-10-18T17:58:26.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb177fe700 1 -- 172.21.15.72:0/3798150066 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0400d4a0 con 0x7fcb180bf060 2023-10-18T17:58:26.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.330+0000 7fcb177fe700 1 -- 172.21.15.72:0/3798150066 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400d820 con 0x7fcb180bf060 2023-10-18T17:58:26.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/3798150066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bf060 msgr2=0x7fcb18068710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:26.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 --2- 172.21.15.72:0/3798150066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bf060 0x7fcb18068710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fcb04009760 tx=0x7fcb04009060).stop 2023-10-18T17:58:26.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/3798150066 shutdown_connections 2023-10-18T17:58:26.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 --2- 172.21.15.72:0/3798150066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bf060 0x7fcb18068710 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:26.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 --2- 172.21.15.72:0/3798150066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb180bec80 0x7fcb180bc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/3798150066 >> 172.21.15.72:0/3798150066 conn(0x7fcb180645a0 msgr2=0x7fcb180649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/3798150066 shutdown_connections 2023-10-18T17:58:26.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/3798150066 wait complete. 2023-10-18T17:58:26.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 Processor -- start 2023-10-18T17:58:26.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.334+0000 7fcb1dc8f700 1 -- start start 2023-10-18T17:58:26.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1dc8f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb18068c00 0x7fcb18155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:26.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1dc8f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bec80 0x7fcb180c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:26.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1dc8f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb180c0ab0 0x7fcb1814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:26.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1dc8f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb181687b0 con 0x7fcb180c0ab0 2023-10-18T17:58:26.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1dc8f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180c0130 con 0x7fcb180bec80 2023-10-18T17:58:26.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1dc8f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18168260 con 0x7fcb18068c00 2023-10-18T17:58:26.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb17fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bec80 0x7fcb180c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:26.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1cc8d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb18068c00 0x7fcb18155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:26.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1d48e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb180c0ab0 0x7fcb1814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:26.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb17fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bec80 0x7fcb180c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51158/0 (socket says 172.21.15.72:51158) 2023-10-18T17:58:26.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb17fff700 1 -- 172.21.15.72:0/733078948 learned_addr learned my addr 172.21.15.72:0/733078948 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:26.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb17fff700 1 -- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb18068c00 msgr2=0x7fcb18155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:26.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb17fff700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb18068c00 0x7fcb18155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:26.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb17fff700 1 -- 172.21.15.72:0/733078948 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb180c0ab0 msgr2=0x7fcb1814f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:26.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb17fff700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb180c0ab0 0x7fcb1814f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:26.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb17fff700 1 -- 172.21.15.72:0/733078948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040092a0 con 0x7fcb180bec80 2023-10-18T17:58:26.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb17fff700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bec80 0x7fcb180c05c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fcb04009d50 tx=0x7fcb04016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:26.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb14ff9700 1 -- 172.21.15.72:0/733078948 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04012770 con 0x7fcb180bec80 2023-10-18T17:58:26.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb14ff9700 1 -- 172.21.15.72:0/733078948 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb040128d0 con 0x7fcb180bec80 2023-10-18T17:58:26.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb180c0f30 con 0x7fcb180bec80 2023-10-18T17:58:26.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb14ff9700 1 -- 172.21.15.72:0/733078948 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04012c70 con 0x7fcb180bec80 2023-10-18T17:58:26.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.338+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb180a0630 con 0x7fcb180bec80 2023-10-18T17:58:26.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.342+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb180466a0 con 0x7fcb180bec80 2023-10-18T17:58:26.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.342+0000 7fcb14ff9700 1 -- 172.21.15.72:0/733078948 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcb0401c630 con 0x7fcb180bec80 2023-10-18T17:58:26.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.342+0000 7fcb14ff9700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcafc05a4e0 0x7fcafc05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:26.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.342+0000 7fcb14ff9700 1 -- 172.21.15.72:0/733078948 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcb04075bf0 con 0x7fcb180bec80 2023-10-18T17:58:26.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.342+0000 7fcb1cc8d700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcafc05a4e0 0x7fcafc05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:26.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.346+0000 7fcb1cc8d700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcafc05a4e0 0x7fcafc05c9a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fcb0c009c30 tx=0x7fcb0c009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:26.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.354+0000 7fcb14ff9700 1 -- 172.21.15.72:0/733078948 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fcb180466a0 con 0x7fcb180bec80 2023-10-18T17:58:26.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:26.558+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcb180466a0 con 0x7fcb180bec80 2023-10-18T17:58:27.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.134+0000 7fcb14ff9700 1 -- 172.21.15.72:0/733078948 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcb180466a0 con 0x7fcb180bec80 2023-10-18T17:58:27.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.138+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcafc05a4e0 msgr2=0x7fcafc05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:27.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.138+0000 7fcb1dc8f700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcafc05a4e0 0x7fcafc05c9a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fcb0c009c30 tx=0x7fcb0c009060).stop 2023-10-18T17:58:27.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.138+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bec80 msgr2=0x7fcb180c05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:27.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.138+0000 7fcb1dc8f700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bec80 0x7fcb180c05c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fcb04009d50 tx=0x7fcb04016000).stop 2023-10-18T17:58:27.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.142+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 shutdown_connections 2023-10-18T17:58:27.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.142+0000 7fcb1dc8f700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcafc05a4e0 0x7fcafc05c9a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:27.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.142+0000 7fcb1dc8f700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcb180c0ab0 0x7fcb1814f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:27.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.142+0000 7fcb1dc8f700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb180bec80 0x7fcb180c05c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:27.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.142+0000 7fcb1dc8f700 1 --2- 172.21.15.72:0/733078948 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcb18068c00 0x7fcb18155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:27.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.142+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 >> 172.21.15.72:0/733078948 conn(0x7fcb180645a0 msgr2=0x7fcb180c1700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:27.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.142+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 shutdown_connections 2023-10-18T17:58:27.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.142+0000 7fcb1dc8f700 1 -- 172.21.15.72:0/733078948 wait complete. 2023-10-18T17:58:27.167 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-18T17:58:27.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb94746700 1 Processor -- start 2023-10-18T17:58:27.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb94746700 1 -- start start 2023-10-18T17:58:27.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb94746700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffb8c0b58b0 0x7ffb8c065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:27.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb94746700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c153cf0 0x7ffb8c067fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:27.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb94746700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c150c60 con 0x7ffb8c066030 2023-10-18T17:58:27.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb94746700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c068f10 con 0x7ffb8c0b58b0 2023-10-18T17:58:27.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb94746700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c066470 con 0x7ffb8c153cf0 2023-10-18T17:58:27.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb92ce3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c153cf0 0x7ffb8c067fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:27.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb91ce1700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ffb8c066030 0x7ffb8c1535e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50530/0 (socket says 172.21.15.72:50530) 2023-10-18T17:58:27.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb92ce3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c153cf0 0x7ffb8c067fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:40824/0 (socket says 172.21.15.72:40824) 2023-10-18T17:58:27.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb91ce1700 1 -- 172.21.15.72:0/3137403184 learned_addr learned my addr 172.21.15.72:0/3137403184 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:27.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.330+0000 7ffb924e2700 1 --2- 172.21.15.72:0/3137403184 >> v2:172.21.15.179:3300/0 conn(0x7ffb8c0b58b0 0x7ffb8c065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb92ce3700 1 -- 172.21.15.72:0/3137403184 >> v2:172.21.15.179:3300/0 conn(0x7ffb8c0b58b0 msgr2=0x7ffb8c065b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb92ce3700 1 --2- 172.21.15.72:0/3137403184 >> v2:172.21.15.179:3300/0 conn(0x7ffb8c0b58b0 0x7ffb8c065b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb914e0700 1 -- 172.21.15.72:0/3137403184 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321106673 0 0) 0x7ffb8c150c60 con 0x7ffb8c066030 2023-10-18T17:58:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb92ce3700 1 -- 172.21.15.72:0/3137403184 >> v1:172.21.15.72:6789/0 conn(0x7ffb8c066030 legacy=0x7ffb8c1535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb92ce3700 1 -- 172.21.15.72:0/3137403184 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0bf6e0 con 0x7ffb8c153cf0 2023-10-18T17:58:27.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb92ce3700 1 --2- 172.21.15.72:0/3137403184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c153cf0 0x7ffb8c067fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ffb84009e60 tx=0x7ffb84009780).ready entity=mon.2 client_cookie=ea2740b1c5774a20 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:27.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb914e0700 1 -- 172.21.15.72:0/3137403184 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400e070 con 0x7ffb8c153cf0 2023-10-18T17:58:27.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb914e0700 1 -- 172.21.15.72:0/3137403184 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8400bb60 con 0x7ffb8c153cf0 2023-10-18T17:58:27.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb914e0700 1 -- 172.21.15.72:0/3137403184 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84012410 con 0x7ffb8c153cf0 2023-10-18T17:58:27.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb94746700 1 -- 172.21.15.72:0/3137403184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c153cf0 msgr2=0x7ffb8c067fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:27.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb94746700 1 --2- 172.21.15.72:0/3137403184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c153cf0 0x7ffb8c067fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ffb84009e60 tx=0x7ffb84009780).stop 2023-10-18T17:58:27.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb94746700 1 -- 172.21.15.72:0/3137403184 shutdown_connections 2023-10-18T17:58:27.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb94746700 1 --2- 172.21.15.72:0/3137403184 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c153cf0 0x7ffb8c067fc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:27.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb94746700 1 --2- 172.21.15.72:0/3137403184 >> v2:172.21.15.179:3300/0 conn(0x7ffb8c0b58b0 0x7ffb8c065b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:27.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb94746700 1 -- 172.21.15.72:0/3137403184 >> 172.21.15.72:0/3137403184 conn(0x7ffb8c0650d0 msgr2=0x7ffb8c05dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:27.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb94746700 1 -- 172.21.15.72:0/3137403184 shutdown_connections 2023-10-18T17:58:27.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.334+0000 7ffb94746700 1 -- 172.21.15.72:0/3137403184 wait complete. 2023-10-18T17:58:27.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.338+0000 7ffb94746700 1 Processor -- start 2023-10-18T17:58:27.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.338+0000 7ffb94746700 1 -- start start 2023-10-18T17:58:27.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb94746700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c066030 0x7ffb8c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:27.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb94746700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c0b58b0 0x7ffb8c14dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:27.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb94746700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb8c153cf0 0x7ffb8c160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:27.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb94746700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0c1260 con 0x7ffb8c153cf0 2023-10-18T17:58:27.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb94746700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c1633d0 con 0x7ffb8c0b58b0 2023-10-18T17:58:27.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb94746700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0bffd0 con 0x7ffb8c066030 2023-10-18T17:58:27.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb91ce1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c0b58b0 0x7ffb8c14dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb924e2700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c066030 0x7ffb8c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb92ce3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb8c153cf0 0x7ffb8c160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb924e2700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c066030 0x7ffb8c0b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:40828/0 (socket says 172.21.15.72:40828) 2023-10-18T17:58:27.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb91ce1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c0b58b0 0x7ffb8c14dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51180/0 (socket says 172.21.15.72:51180) 2023-10-18T17:58:27.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb924e2700 1 -- 172.21.15.72:0/1136195620 learned_addr learned my addr 172.21.15.72:0/1136195620 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:27.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb924e2700 1 -- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c0b58b0 msgr2=0x7ffb8c14dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:27.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb924e2700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c0b58b0 0x7ffb8c14dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:27.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb924e2700 1 -- 172.21.15.72:0/1136195620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb8c153cf0 msgr2=0x7ffb8c160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:27.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb924e2700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb8c153cf0 0x7ffb8c160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb924e2700 1 -- 172.21.15.72:0/1136195620 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb84009a20 con 0x7ffb8c066030 2023-10-18T17:58:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb91ce1700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c0b58b0 0x7ffb8c14dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb924e2700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c066030 0x7ffb8c0b4e40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ffb8800e960 tx=0x7ffb88009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:27.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb827fc700 1 -- 172.21.15.72:0/1136195620 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800c680 con 0x7ffb8c066030 2023-10-18T17:58:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb827fc700 1 -- 172.21.15.72:0/1136195620 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8800c7e0 con 0x7ffb8c066030 2023-10-18T17:58:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c0c0270 con 0x7ffb8c066030 2023-10-18T17:58:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.342+0000 7ffb827fc700 1 -- 172.21.15.72:0/1136195620 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800cb60 con 0x7ffb8c066030 2023-10-18T17:58:27.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.346+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c0c0810 con 0x7ffb8c066030 2023-10-18T17:58:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.346+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c0c32c0 con 0x7ffb8c066030 2023-10-18T17:58:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.350+0000 7ffb827fc700 1 -- 172.21.15.72:0/1136195620 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ffb8801b400 con 0x7ffb8c066030 2023-10-18T17:58:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.350+0000 7ffb827fc700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffb7005a560 0x7ffb7005ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.350+0000 7ffb827fc700 1 -- 172.21.15.72:0/1136195620 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ffb88074c90 con 0x7ffb8c066030 2023-10-18T17:58:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.350+0000 7ffb91ce1700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffb7005a560 0x7ffb7005ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:27.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.350+0000 7ffb91ce1700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffb7005a560 0x7ffb7005ca20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7ffb8c14d650 tx=0x7ffb7c007bb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:27.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.362+0000 7ffb827fc700 1 -- 172.21.15.72:0/1136195620 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ffb8c0c32c0 con 0x7ffb8c066030 2023-10-18T17:58:27.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:27.570+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7ffb8c0c32c0 con 0x7ffb8c066030 2023-10-18T17:58:28.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.142+0000 7ffb827fc700 1 -- 172.21.15.72:0/1136195620 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 115+0+0 (secure 0 0 0) 0x7ffb8c0c32c0 con 0x7ffb8c066030 2023-10-18T17:58:28.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffb7005a560 msgr2=0x7ffb7005ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffb7005a560 0x7ffb7005ca20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7ffb8c14d650 tx=0x7ffb7c007bb0).stop 2023-10-18T17:58:28.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c066030 msgr2=0x7ffb8c0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c066030 0x7ffb8c0b4e40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ffb8800e960 tx=0x7ffb88009060).stop 2023-10-18T17:58:28.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 shutdown_connections 2023-10-18T17:58:28.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffb7005a560 0x7ffb7005ca20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb8c153cf0 0x7ffb8c160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c0b58b0 0x7ffb8c14dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 --2- 172.21.15.72:0/1136195620 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffb8c066030 0x7ffb8c0b4e40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 >> 172.21.15.72:0/1136195620 conn(0x7ffb8c0650d0 msgr2=0x7ffb8c0620d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:28.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 shutdown_connections 2023-10-18T17:58:28.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.146+0000 7ffb94746700 1 -- 172.21.15.72:0/1136195620 wait complete. 2023-10-18T17:58:28.174 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T17:58:28.174 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T17:58:28.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.342+0000 7f5f22d47700 1 Processor -- start 2023-10-18T17:58:28.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.342+0000 7f5f22d47700 1 -- start start 2023-10-18T17:58:28.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.342+0000 7f5f22d47700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c0bf340 0x7f5f1c0bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.342+0000 7f5f22d47700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5f1c13d890 0x7f5f1c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.342+0000 7f5f22d47700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c068ea0 con 0x7f5f1c0bcaf0 2023-10-18T17:58:28.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.342+0000 7f5f22d47700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c154220 con 0x7f5f1c13d890 2023-10-18T17:58:28.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.342+0000 7f5f22d47700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c157970 con 0x7f5f1c0bf340 2023-10-18T17:58:28.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f21544700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f5f1c0bcaf0 0x7f5f1c13d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50532/0 (socket says 172.21.15.72:50532) 2023-10-18T17:58:28.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f21544700 1 -- 172.21.15.72:0/2362484859 learned_addr learned my addr 172.21.15.72:0/2362484859 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:28.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f21d45700 1 --2- 172.21.15.72:0/2362484859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c0bf340 0x7f5f1c0bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f22546700 1 --2- 172.21.15.72:0/2362484859 >> v2:172.21.15.179:3300/0 conn(0x7f5f1c13d890 0x7f5f1c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f21d45700 1 -- 172.21.15.72:0/2362484859 >> v2:172.21.15.179:3300/0 conn(0x7f5f1c13d890 msgr2=0x7f5f1c0689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:28.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f21d45700 1 --2- 172.21.15.72:0/2362484859 >> v2:172.21.15.179:3300/0 conn(0x7f5f1c13d890 0x7f5f1c0689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f20d43700 1 -- 172.21.15.72:0/2362484859 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580441554 0 0) 0x7f5f1c068ea0 con 0x7f5f1c0bcaf0 2023-10-18T17:58:28.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f21d45700 1 -- 172.21.15.72:0/2362484859 >> v1:172.21.15.72:6789/0 conn(0x7f5f1c0bcaf0 legacy=0x7f5f1c13d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f21d45700 1 -- 172.21.15.72:0/2362484859 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c157bf0 con 0x7f5f1c0bf340 2023-10-18T17:58:28.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f21d45700 1 --2- 172.21.15.72:0/2362484859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c0bf340 0x7f5f1c0bf720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f5f18009760 tx=0x7f5f18009060).ready entity=mon.2 client_cookie=404226766ccec89e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:28.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f20d43700 1 -- 172.21.15.72:0/2362484859 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800b590 con 0x7f5f1c0bf340 2023-10-18T17:58:28.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f20d43700 1 -- 172.21.15.72:0/2362484859 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1800b6f0 con 0x7f5f1c0bf340 2023-10-18T17:58:28.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f20d43700 1 -- 172.21.15.72:0/2362484859 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800ba70 con 0x7f5f1c0bf340 2023-10-18T17:58:28.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f22d47700 1 -- 172.21.15.72:0/2362484859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c0bf340 msgr2=0x7f5f1c0bf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f22d47700 1 --2- 172.21.15.72:0/2362484859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c0bf340 0x7f5f1c0bf720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f5f18009760 tx=0x7f5f18009060).stop 2023-10-18T17:58:28.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f22d47700 1 -- 172.21.15.72:0/2362484859 shutdown_connections 2023-10-18T17:58:28.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f22d47700 1 --2- 172.21.15.72:0/2362484859 >> v2:172.21.15.179:3300/0 conn(0x7f5f1c13d890 0x7f5f1c0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f22d47700 1 --2- 172.21.15.72:0/2362484859 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c0bf340 0x7f5f1c0bf720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f22d47700 1 -- 172.21.15.72:0/2362484859 >> 172.21.15.72:0/2362484859 conn(0x7f5f1c05ddc0 msgr2=0x7f5f1c05e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f22d47700 1 -- 172.21.15.72:0/2362484859 shutdown_connections 2023-10-18T17:58:28.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.346+0000 7f5f22d47700 1 -- 172.21.15.72:0/2362484859 wait complete. 2023-10-18T17:58:28.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.350+0000 7f5f22d47700 1 Processor -- start 2023-10-18T17:58:28.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.350+0000 7f5f22d47700 1 -- start start 2023-10-18T17:58:28.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.350+0000 7f5f22d47700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f1c0bcaf0 0x7f5f1c152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.350+0000 7f5f22d47700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f1c0bf340 0x7f5f1c156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.350+0000 7f5f22d47700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c13d890 0x7f5f1c135fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.350+0000 7f5f22d47700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0b9cc0 con 0x7f5f1c0bf340 2023-10-18T17:58:28.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.350+0000 7f5f22d47700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c152b50 con 0x7f5f1c0bcaf0 2023-10-18T17:58:28.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.350+0000 7f5f22d47700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c164030 con 0x7f5f1c13d890 2023-10-18T17:58:28.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22546700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c13d890 0x7f5f1c135fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f21d45700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f1c0bcaf0 0x7f5f1c152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22546700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c13d890 0x7f5f1c135fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:40834/0 (socket says 172.21.15.72:40834) 2023-10-18T17:58:28.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22546700 1 -- 172.21.15.72:0/923065258 learned_addr learned my addr 172.21.15.72:0/923065258 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:28.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f21544700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f1c0bf340 0x7f5f1c156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22546700 1 -- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f1c0bcaf0 msgr2=0x7f5f1c152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22546700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f1c0bcaf0 0x7f5f1c152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22546700 1 -- 172.21.15.72:0/923065258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f1c0bf340 msgr2=0x7f5f1c156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22546700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f1c0bf340 0x7f5f1c156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22546700 1 -- 172.21.15.72:0/923065258 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f180092a0 con 0x7f5f1c13d890 2023-10-18T17:58:28.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f21d45700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f1c0bcaf0 0x7f5f1c152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:28.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22546700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c13d890 0x7f5f1c135fd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5f14009da0 tx=0x7f5f1400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:28.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f11ffb700 1 -- 172.21.15.72:0/923065258 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400caa0 con 0x7f5f1c13d890 2023-10-18T17:58:28.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f11ffb700 1 -- 172.21.15.72:0/923065258 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1400f040 con 0x7f5f1c13d890 2023-10-18T17:58:28.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f11ffb700 1 -- 172.21.15.72:0/923065258 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400ce10 con 0x7f5f1c13d890 2023-10-18T17:58:28.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c1642b0 con 0x7f5f1c13d890 2023-10-18T17:58:28.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.354+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c152490 con 0x7f5f1c13d890 2023-10-18T17:58:28.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.358+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c152c90 con 0x7f5f1c13d890 2023-10-18T17:58:28.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.358+0000 7f5f11ffb700 1 -- 172.21.15.72:0/923065258 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5f14013610 con 0x7f5f1c13d890 2023-10-18T17:58:28.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.358+0000 7f5f11ffb700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5f0005a510 0x7f5f0005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.358+0000 7f5f11ffb700 1 -- 172.21.15.72:0/923065258 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5f14074f80 con 0x7f5f1c13d890 2023-10-18T17:58:28.364 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.358+0000 7f5f21d45700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5f0005a510 0x7f5f0005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.364 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.362+0000 7f5f21d45700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5f0005a510 0x7f5f0005c9d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5f1800b350 tx=0x7f5f18017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:28.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.370+0000 7f5f11ffb700 1 -- 172.21.15.72:0/923065258 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f5f1c152c90 con 0x7f5f1c13d890 2023-10-18T17:58:28.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.574+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f1c152c90 con 0x7f5f1c13d890 2023-10-18T17:58:28.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.574+0000 7f5f11ffb700 1 -- 172.21.15.72:0/923065258 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6001 (secure 0 0 0) 0x7f5f1c152c90 con 0x7f5f1c13d890 2023-10-18T17:58:28.579 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:28.579 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4215,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6836/3359824101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3359824101},{"type":"v1","addr":"172.21.15.72:6836","nonce":3359824101}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3056786712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3056786712},{"type":"v1","addr":"172.21.15.179:6835","nonce":3056786712}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/4028468466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":4028468466},{"type":"v1","addr":"172.21.15.179:6839","nonce":4028468466}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1416523495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6835","nonce":1416523495},{"type":"v1","addr":"172.21.15.72:6837","nonce":1416523495}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/814982392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":814982392},{"type":"v1","addr":"172.21.15.179:6837","nonce":814982392}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T17:58:23.091532+0000","modified":"2023-10-18T17:58:28.134516+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"a","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.72:6833/247898707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":247898707},{"type":"v1","addr":"172.21.15.72:6833","nonce":247898707}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-18T17:58:28.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5f0005a510 msgr2=0x7f5f0005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5f0005a510 0x7f5f0005c9d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5f1800b350 tx=0x7f5f18017000).stop 2023-10-18T17:58:28.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c13d890 msgr2=0x7f5f1c135fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c13d890 0x7f5f1c135fd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5f14009da0 tx=0x7f5f1400b040).stop 2023-10-18T17:58:28.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 shutdown_connections 2023-10-18T17:58:28.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5f0005a510 0x7f5f0005c9d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5f1c13d890 0x7f5f1c135fd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5f1c0bf340 0x7f5f1c156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 --2- 172.21.15.72:0/923065258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f1c0bcaf0 0x7f5f1c152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 >> 172.21.15.72:0/923065258 conn(0x7f5f1c05ddc0 msgr2=0x7f5f1c067fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:28.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 shutdown_connections 2023-10-18T17:58:28.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.578+0000 7f5f22d47700 1 -- 172.21.15.72:0/923065258 wait complete. 2023-10-18T17:58:28.588 INFO:teuthology.orchestra.run.smithi072.stderr:dumped fsmap epoch 8 2023-10-18T17:58:28.602 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:28.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.758+0000 7fb64a907700 1 Processor -- start 2023-10-18T17:58:28.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.758+0000 7fb64a907700 1 -- start start 2023-10-18T17:58:28.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb64a907700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb6440bf340 0x7fb6440bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb64a907700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb6440bcea0 0x7fb644066240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb64a907700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb644153520 con 0x7fb644066730 2023-10-18T17:58:28.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb64a907700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb644068150 con 0x7fb6440bf340 2023-10-18T17:58:28.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb64a907700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb644068430 con 0x7fb6440bcea0 2023-10-18T17:58:28.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb649905700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb6440bf340 0x7fb6440bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb649104700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb6440bcea0 0x7fb644066240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb649905700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb6440bf340 0x7fb6440bf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58776/0 (socket says 172.21.15.72:58776) 2023-10-18T17:58:28.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb649905700 1 -- 172.21.15.72:0/2426597729 learned_addr learned my addr 172.21.15.72:0/2426597729 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:28.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb649905700 1 -- 172.21.15.72:0/2426597729 >> v2:172.21.15.179:3300/0 conn(0x7fb6440bcea0 msgr2=0x7fb644066240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:28.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb649905700 1 --2- 172.21.15.72:0/2426597729 >> v2:172.21.15.179:3300/0 conn(0x7fb6440bcea0 0x7fb644066240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb649905700 1 -- 172.21.15.72:0/2426597729 >> v1:172.21.15.179:6789/0 conn(0x7fb644066730 legacy=0x7fb6441539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb649905700 1 -- 172.21.15.72:0/2426597729 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb644068710 con 0x7fb6440bf340 2023-10-18T17:58:28.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb649905700 1 --2- 172.21.15.72:0/2426597729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 0x7fb6440bf720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb6440bfc10 tx=0x7fb6400090f0).ready entity=mon.0 client_cookie=756da1668b49dbd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:28.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb648903700 1 -- 172.21.15.72:0/2426597729 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64000b5b0 con 0x7fb6440bf340 2023-10-18T17:58:28.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb648903700 1 -- 172.21.15.72:0/2426597729 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64000b710 con 0x7fb6440bf340 2023-10-18T17:58:28.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.762+0000 7fb648903700 1 -- 172.21.15.72:0/2426597729 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64000ba90 con 0x7fb6440bf340 2023-10-18T17:58:28.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 -- 172.21.15.72:0/2426597729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 msgr2=0x7fb6440bf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 --2- 172.21.15.72:0/2426597729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 0x7fb6440bf720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb6440bfc10 tx=0x7fb6400090f0).stop 2023-10-18T17:58:28.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 -- 172.21.15.72:0/2426597729 shutdown_connections 2023-10-18T17:58:28.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 --2- 172.21.15.72:0/2426597729 >> v2:172.21.15.179:3300/0 conn(0x7fb6440bcea0 0x7fb644066240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 --2- 172.21.15.72:0/2426597729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 0x7fb6440bf720 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 -- 172.21.15.72:0/2426597729 >> 172.21.15.72:0/2426597729 conn(0x7fb64405ddc0 msgr2=0x7fb64405e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:28.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 -- 172.21.15.72:0/2426597729 shutdown_connections 2023-10-18T17:58:28.772 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 -- 172.21.15.72:0/2426597729 wait complete. 2023-10-18T17:58:28.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 Processor -- start 2023-10-18T17:58:28.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.766+0000 7fb64a907700 1 -- start start 2023-10-18T17:58:28.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a907700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb644066730 0x7fb6440bc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a907700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6440bcea0 0x7fb644163c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a907700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 0x7fb6441606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a907700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440c2f80 con 0x7fb6440bf340 2023-10-18T17:58:28.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a907700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb644163770 con 0x7fb6440bcea0 2023-10-18T17:58:28.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a907700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb644154bd0 con 0x7fb644066730 2023-10-18T17:58:28.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649905700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb644066730 0x7fb6440bc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649104700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6440bcea0 0x7fb644163c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649905700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb644066730 0x7fb6440bc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:40840/0 (socket says 172.21.15.72:40840) 2023-10-18T17:58:28.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649905700 1 -- 172.21.15.72:0/3370569369 learned_addr learned my addr 172.21.15.72:0/3370569369 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:28.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a106700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 0x7fb6441606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649104700 1 -- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb644066730 msgr2=0x7fb6440bc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649104700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb644066730 0x7fb6440bc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649104700 1 -- 172.21.15.72:0/3370569369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 msgr2=0x7fb6441606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649104700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 0x7fb6441606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649104700 1 -- 172.21.15.72:0/3370569369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640009370 con 0x7fb6440bcea0 2023-10-18T17:58:28.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a106700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 0x7fb6441606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:28.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb649104700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6440bcea0 0x7fb644163c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb63400e900 tx=0x7fb634009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:28.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb639ffb700 1 -- 172.21.15.72:0/3370569369 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634012600 con 0x7fb6440bcea0 2023-10-18T17:58:28.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb639ffb700 1 -- 172.21.15.72:0/3370569369 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb634012760 con 0x7fb6440bcea0 2023-10-18T17:58:28.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb644154eb0 con 0x7fb6440bcea0 2023-10-18T17:58:28.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb639ffb700 1 -- 172.21.15.72:0/3370569369 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634012ae0 con 0x7fb6440bcea0 2023-10-18T17:58:28.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.770+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb644163400 con 0x7fb6440bcea0 2023-10-18T17:58:28.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.774+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb644046f00 con 0x7fb6440bcea0 2023-10-18T17:58:28.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.774+0000 7fb639ffb700 1 -- 172.21.15.72:0/3370569369 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb63401b400 con 0x7fb6440bcea0 2023-10-18T17:58:28.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.774+0000 7fb639ffb700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb62805a510 0x7fb62805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:28.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.774+0000 7fb639ffb700 1 -- 172.21.15.72:0/3370569369 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb634074ca0 con 0x7fb6440bcea0 2023-10-18T17:58:28.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.774+0000 7fb649905700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb62805a510 0x7fb62805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:28.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.778+0000 7fb649905700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb62805a510 0x7fb62805c9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb64000c010 tx=0x7fb640017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:28.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.782+0000 7fb639ffb700 1 -- 172.21.15.72:0/3370569369 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fb6340483e0 con 0x7fb6440bcea0 2023-10-18T17:58:28.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.950+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6441554c0 con 0x7fb6440bcea0 2023-10-18T17:58:28.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.954+0000 7fb639ffb700 1 -- 172.21.15.72:0/3370569369 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7fb63404ba00 con 0x7fb6440bcea0 2023-10-18T17:58:28.957 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:28.958 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":22,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:24.095038+0000","last_up_change":"2023-10-18T17:58:14.550597+0000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T17:58:13.627096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T17:58:15.732786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T17:58:21.178314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T17:58:22.112742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":33232},{"type":"v1","addr":"172.21.15.72:6810","nonce":33232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6813","nonce":33232},{"type":"v1","addr":"172.21.15.72:6815","nonce":33232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":33232},{"type":"v1","addr":"172.21.15.72:6826","nonce":33232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":33232},{"type":"v1","addr":"172.21.15.72:6820","nonce":33232}]},"public_addr":"172.21.15.72:6810/33232","cluster_addr":"172.21.15.72:6815/33232","heartbeat_back_addr":"172.21.15.72:6826/33232","heartbeat_front_addr":"172.21.15.72:6820/33232","state":["exists","up"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6809","nonce":33229},{"type":"v1","addr":"172.21.15.72:6812","nonce":33229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":33229},{"type":"v1","addr":"172.21.15.72:6819","nonce":33229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":33229},{"type":"v1","addr":"172.21.15.72:6831","nonce":33229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":33229},{"type":"v1","addr":"172.21.15.72:6825","nonce":33229}]},"public_addr":"172.21.15.72:6812/33229","cluster_addr":"172.21.15.72:6819/33229","heartbeat_back_addr":"172.21.15.72:6831/33229","heartbeat_front_addr":"172.21.15.72:6825/33229","state":["exists","up"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":33231},{"type":"v1","addr":"172.21.15.72:6801","nonce":33231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":33231},{"type":"v1","addr":"172.21.15.72:6803","nonce":33231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":33231},{"type":"v1","addr":"172.21.15.72:6807","nonce":33231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":33231},{"type":"v1","addr":"172.21.15.72:6805","nonce":33231}]},"public_addr":"172.21.15.72:6801/33231","cluster_addr":"172.21.15.72:6803/33231","heartbeat_back_addr":"172.21.15.72:6807/33231","heartbeat_front_addr":"172.21.15.72:6805/33231","state":["exists","up"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":33228},{"type":"v1","addr":"172.21.15.72:6814","nonce":33228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":33228},{"type":"v1","addr":"172.21.15.72:6821","nonce":33228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":33228},{"type":"v1","addr":"172.21.15.72:6830","nonce":33228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":33228},{"type":"v1","addr":"172.21.15.72:6827","nonce":33228}]},"public_addr":"172.21.15.72:6814/33228","cluster_addr":"172.21.15.72:6821/33228","heartbeat_back_addr":"172.21.15.72:6830/33228","heartbeat_front_addr":"172.21.15.72:6827/33228","state":["exists","up"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":34034},{"type":"v1","addr":"172.21.15.179:6809","nonce":34034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":34034},{"type":"v1","addr":"172.21.15.179:6813","nonce":34034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":34034},{"type":"v1","addr":"172.21.15.179:6821","nonce":34034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":34034},{"type":"v1","addr":"172.21.15.179:6817","nonce":34034}]},"public_addr":"172.21.15.179:6809/34034","cluster_addr":"172.21.15.179:6813/34034","heartbeat_back_addr":"172.21.15.179:6821/34034","heartbeat_front_addr":"172.21.15.179:6817/34034","state":["exists","up"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":34044},{"type":"v1","addr":"172.21.15.179:6801","nonce":34044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":34044},{"type":"v1","addr":"172.21.15.179:6803","nonce":34044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":34044},{"type":"v1","addr":"172.21.15.179:6807","nonce":34044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":34044},{"type":"v1","addr":"172.21.15.179:6805","nonce":34044}]},"public_addr":"172.21.15.179:6801/34044","cluster_addr":"172.21.15.179:6803/34044","heartbeat_back_addr":"172.21.15.179:6807/34044","heartbeat_front_addr":"172.21.15.179:6805/34044","state":["exists","up"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":34036},{"type":"v1","addr":"172.21.15.179:6825","nonce":34036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":34036},{"type":"v1","addr":"172.21.15.179:6827","nonce":34036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":34036},{"type":"v1","addr":"172.21.15.179:6831","nonce":34036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":34036},{"type":"v1","addr":"172.21.15.179:6829","nonce":34036}]},"public_addr":"172.21.15.179:6825/34036","cluster_addr":"172.21.15.179:6827/34036","heartbeat_back_addr":"172.21.15.179:6831/34036","heartbeat_front_addr":"172.21.15.179:6829/34036","state":["exists","up"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":34035},{"type":"v1","addr":"172.21.15.179:6812","nonce":34035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":34035},{"type":"v1","addr":"172.21.15.179:6816","nonce":34035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":34035},{"type":"v1","addr":"172.21.15.179:6823","nonce":34035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":34035},{"type":"v1","addr":"172.21.15.179:6820","nonce":34035}]},"public_addr":"172.21.15.179:6812/34035","cluster_addr":"172.21.15.179:6816/34035","heartbeat_back_addr":"172.21.15.179:6823/34035","heartbeat_front_addr":"172.21.15.179:6820/34035","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.697876+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.661119+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.579136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.792036+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.069244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:11.971339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.176699+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.197004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:28.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb62805a510 msgr2=0x7fb62805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb62805a510 0x7fb62805c9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb64000c010 tx=0x7fb640017000).stop 2023-10-18T17:58:28.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6440bcea0 msgr2=0x7fb644163c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:28.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6440bcea0 0x7fb644163c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb63400e900 tx=0x7fb634009060).stop 2023-10-18T17:58:28.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 shutdown_connections 2023-10-18T17:58:28.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb62805a510 0x7fb62805c9d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6440bf340 0x7fb6441606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6440bcea0 0x7fb644163c00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 --2- 172.21.15.72:0/3370569369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb644066730 0x7fb6440bc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:28.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 >> 172.21.15.72:0/3370569369 conn(0x7fb64405ddc0 msgr2=0x7fb64414dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:28.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 shutdown_connections 2023-10-18T17:58:28.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:28.958+0000 7fb64a907700 1 -- 172.21.15.72:0/3370569369 wait complete. 2023-10-18T17:58:28.980 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T17:58:28.981 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T17:58:29.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.134+0000 7f5cd8369700 1 Processor -- start 2023-10-18T17:58:29.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.134+0000 7f5cd8369700 1 -- start start 2023-10-18T17:58:29.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd8369700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f5cd01499b0 0x7f5cd00bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd8369700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f5cd00bd270 0x7f5cd00bdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd8369700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd0068520 con 0x7f5cd00b5390 2023-10-18T17:58:29.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd8369700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd00690c0 con 0x7f5cd01499b0 2023-10-18T17:58:29.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd8369700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd0150710 con 0x7f5cd00bd270 2023-10-18T17:58:29.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd5904700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f5cd00b5390 0x7f5cd00689b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:60726/0 (socket says 172.21.15.72:60726) 2023-10-18T17:58:29.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd5904700 1 -- 172.21.15.72:0/83094324 learned_addr learned my addr 172.21.15.72:0/83094324 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:29.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd6105700 1 --2- 172.21.15.72:0/83094324 >> v2:172.21.15.72:3300/0 conn(0x7f5cd01499b0 0x7f5cd00bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd6906700 1 --2- 172.21.15.72:0/83094324 >> v2:172.21.15.179:3300/0 conn(0x7f5cd00bd270 0x7f5cd00bdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd5103700 1 -- 172.21.15.72:0/83094324 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791292507 0 0) 0x7f5cd0068520 con 0x7f5cd00b5390 2023-10-18T17:58:29.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd5103700 1 -- 172.21.15.72:0/83094324 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cbc0033f0 con 0x7f5cd00b5390 2023-10-18T17:58:29.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd6906700 1 -- 172.21.15.72:0/83094324 >> v2:172.21.15.72:3300/0 conn(0x7f5cd01499b0 msgr2=0x7f5cd00bf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:29.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd6906700 1 --2- 172.21.15.72:0/83094324 >> v2:172.21.15.72:3300/0 conn(0x7f5cd01499b0 0x7f5cd00bf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd6906700 1 -- 172.21.15.72:0/83094324 >> v1:172.21.15.179:6789/0 conn(0x7f5cd00b5390 legacy=0x7f5cd00689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd6906700 1 -- 172.21.15.72:0/83094324 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd0149d90 con 0x7f5cd00bd270 2023-10-18T17:58:29.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd6105700 1 --2- 172.21.15.72:0/83094324 >> v2:172.21.15.72:3300/0 conn(0x7f5cd01499b0 0x7f5cd00bf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:29.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.138+0000 7f5cd6906700 1 --2- 172.21.15.72:0/83094324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cd00bd270 0x7f5cd00bdd60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5cd00beb80 tx=0x7f5cc8009780).ready entity=mon.1 client_cookie=3ebad596bd4ec64b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:29.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd5103700 1 -- 172.21.15.72:0/83094324 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc800e070 con 0x7f5cd00bd270 2023-10-18T17:58:29.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd5103700 1 -- 172.21.15.72:0/83094324 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc800bb40 con 0x7f5cd00bd270 2023-10-18T17:58:29.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd5103700 1 -- 172.21.15.72:0/83094324 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8012410 con 0x7f5cd00bd270 2023-10-18T17:58:29.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 -- 172.21.15.72:0/83094324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cd00bd270 msgr2=0x7f5cd00bdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 --2- 172.21.15.72:0/83094324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cd00bd270 0x7f5cd00bdd60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5cd00beb80 tx=0x7f5cc8009780).stop 2023-10-18T17:58:29.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 -- 172.21.15.72:0/83094324 shutdown_connections 2023-10-18T17:58:29.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 --2- 172.21.15.72:0/83094324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cd00bd270 0x7f5cd00bdd60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 --2- 172.21.15.72:0/83094324 >> v2:172.21.15.72:3300/0 conn(0x7f5cd01499b0 0x7f5cd00bf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 -- 172.21.15.72:0/83094324 >> 172.21.15.72:0/83094324 conn(0x7f5cd00620d0 msgr2=0x7f5cd00624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:29.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 -- 172.21.15.72:0/83094324 shutdown_connections 2023-10-18T17:58:29.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 -- 172.21.15.72:0/83094324 wait complete. 2023-10-18T17:58:29.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 Processor -- start 2023-10-18T17:58:29.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.142+0000 7f5cd8369700 1 -- start start 2023-10-18T17:58:29.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd8369700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cd00b5390 0x7f5cd00b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd8369700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cd00bd270 0x7f5cd00b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd8369700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5cd01499b0 0x7f5cd014e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd8369700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00c2a10 con 0x7f5cd00bd270 2023-10-18T17:58:29.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd8369700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00c2f20 con 0x7f5cd00b5390 2023-10-18T17:58:29.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd8369700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd0160900 con 0x7f5cd01499b0 2023-10-18T17:58:29.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd5904700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cd00bd270 0x7f5cd00b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd5904700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cd00bd270 0x7f5cd00b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58806/0 (socket says 172.21.15.72:58806) 2023-10-18T17:58:29.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd5904700 1 -- 172.21.15.72:0/1938201189 learned_addr learned my addr 172.21.15.72:0/1938201189 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:29.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd6105700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cd00b5390 0x7f5cd00b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd6906700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5cd01499b0 0x7f5cd014e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd5904700 1 -- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5cd01499b0 msgr2=0x7f5cd014e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd5904700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5cd01499b0 0x7f5cd014e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd5904700 1 -- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cd00b5390 msgr2=0x7f5cd00b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd5904700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cd00b5390 0x7f5cd00b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd5904700 1 -- 172.21.15.72:0/1938201189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc0092a0 con 0x7f5cd00bd270 2023-10-18T17:58:29.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.146+0000 7f5cd5904700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cd00bd270 0x7f5cd00b7560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5cc000c5c0 tx=0x7f5cc0007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:29.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.150+0000 7f5cc67fc700 1 -- 172.21.15.72:0/1938201189 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00072e0 con 0x7f5cd00bd270 2023-10-18T17:58:29.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.150+0000 7f5cc67fc700 1 -- 172.21.15.72:0/1938201189 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc0007440 con 0x7f5cd00bd270 2023-10-18T17:58:29.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.150+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc8009a20 con 0x7f5cd00bd270 2023-10-18T17:58:29.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.150+0000 7f5cc67fc700 1 -- 172.21.15.72:0/1938201189 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00057a0 con 0x7f5cd00bd270 2023-10-18T17:58:29.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.150+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd01602b0 con 0x7f5cd00bd270 2023-10-18T17:58:29.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.150+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd0160110 con 0x7f5cd00bd270 2023-10-18T17:58:29.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.150+0000 7f5cc67fc700 1 -- 172.21.15.72:0/1938201189 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5cc0005bd0 con 0x7f5cd00bd270 2023-10-18T17:58:29.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.154+0000 7f5cc67fc700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5cb405a4e0 0x7f5cb405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.154+0000 7f5cc67fc700 1 -- 172.21.15.72:0/1938201189 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5cc0074000 con 0x7f5cd00bd270 2023-10-18T17:58:29.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.154+0000 7f5cd6105700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5cb405a4e0 0x7f5cb405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.154+0000 7f5cd6105700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5cb405a4e0 0x7f5cb405c9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5ccc009d20 tx=0x7f5ccc009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:29.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.158+0000 7f5cc67fc700 1 -- 172.21.15.72:0/1938201189 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f5cd0160110 con 0x7f5cd00bd270 2023-10-18T17:58:29.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.322+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 --> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5cd0160110 con 0x7f5cb405a4e0 2023-10-18T17:58:29.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.338+0000 7f5cc67fc700 1 -- 172.21.15.72:0/1938201189 <== mgr.4102 v2:172.21.15.179:6832/33696 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336124 (secure 0 0 0) 0x7f5cd0160110 con 0x7f5cb405a4e0 2023-10-18T17:58:29.346 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:29.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5cb405a4e0 msgr2=0x7f5cb405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5cb405a4e0 0x7f5cb405c9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5ccc009d20 tx=0x7f5ccc009060).stop 2023-10-18T17:58:29.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cd00bd270 msgr2=0x7f5cd00b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cd00bd270 0x7f5cd00b7560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5cc000c5c0 tx=0x7f5cc0007c80).stop 2023-10-18T17:58:29.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 shutdown_connections 2023-10-18T17:58:29.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f5cb405a4e0 0x7f5cb405c9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f5cd01499b0 0x7f5cd014e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5cd00bd270 0x7f5cd00b7560 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 --2- 172.21.15.72:0/1938201189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cd00b5390 0x7f5cd00b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 >> 172.21.15.72:0/1938201189 conn(0x7f5cd00620d0 msgr2=0x7f5cd0065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:29.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 shutdown_connections 2023-10-18T17:58:29.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.346+0000 7f5cd8369700 1 -- 172.21.15.72:0/1938201189 wait complete. 2023-10-18T17:58:29.354 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-10-18T17:58:29.371 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T17:58:28.468052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378468,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596252,"statfs":{"total":773094113280,"available":770658562048,"internally_reserved":0,"allocated":1740800,"data_stored":643085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129427+0000","last_change":"2023-10-18T17:58:23.125789+0000","last_active":"2023-10-18T17:58:23.129427+0000","last_peered":"2023-10-18T17:58:23.129427+0000","last_clean":"2023-10-18T17:58:23.129427+0000","last_became_active":"2023-10-18T17:58:23.123952+0000","last_became_peered":"2023-10-18T17:58:23.123952+0000","last_unstale":"2023-10-18T17:58:23.129427+0000","last_undegraded":"2023-10-18T17:58:23.129427+0000","last_fullsized":"2023-10-18T17:58:23.129427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.682906+0000","last_change":"2023-10-18T17:58:22.680518+0000","last_active":"2023-10-18T17:58:22.682906+0000","last_peered":"2023-10-18T17:58:22.682906+0000","last_clean":"2023-10-18T17:58:22.682906+0000","last_became_active":"2023-10-18T17:58:22.679123+0000","last_became_peered":"2023-10-18T17:58:22.679123+0000","last_unstale":"2023-10-18T17:58:22.682906+0000","last_undegraded":"2023-10-18T17:58:22.682906+0000","last_fullsized":"2023-10-18T17:58:22.682906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121498+0000","last_change":"2023-10-18T17:58:23.121498+0000","last_active":"2023-10-18T17:58:23.121498+0000","last_peered":"2023-10-18T17:58:23.121498+0000","last_clean":"2023-10-18T17:58:23.121498+0000","last_became_active":"2023-10-18T17:58:23.121129+0000","last_became_peered":"2023-10-18T17:58:23.121129+0000","last_unstale":"2023-10-18T17:58:23.121498+0000","last_undegraded":"2023-10-18T17:58:23.121498+0000","last_fullsized":"2023-10-18T17:58:23.121498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114330+0000","last_change":"2023-10-18T17:58:22.701312+0000","last_active":"2023-10-18T17:58:23.114330+0000","last_peered":"2023-10-18T17:58:23.114330+0000","last_clean":"2023-10-18T17:58:23.114330+0000","last_became_active":"2023-10-18T17:58:22.701013+0000","last_became_peered":"2023-10-18T17:58:22.701013+0000","last_unstale":"2023-10-18T17:58:23.114330+0000","last_undegraded":"2023-10-18T17:58:23.114330+0000","last_fullsized":"2023-10-18T17:58:23.114330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120622+0000","last_change":"2023-10-18T17:58:22.690786+0000","last_active":"2023-10-18T17:58:23.120622+0000","last_peered":"2023-10-18T17:58:23.120622+0000","last_clean":"2023-10-18T17:58:23.120622+0000","last_became_active":"2023-10-18T17:58:22.690530+0000","last_became_peered":"2023-10-18T17:58:22.690530+0000","last_unstale":"2023-10-18T17:58:23.120622+0000","last_undegraded":"2023-10-18T17:58:23.120622+0000","last_fullsized":"2023-10-18T17:58:23.120622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121219+0000","last_change":"2023-10-18T17:58:23.112227+0000","last_active":"2023-10-18T17:58:23.121219+0000","last_peered":"2023-10-18T17:58:23.121219+0000","last_clean":"2023-10-18T17:58:23.121219+0000","last_became_active":"2023-10-18T17:58:23.111236+0000","last_became_peered":"2023-10-18T17:58:23.111236+0000","last_unstale":"2023-10-18T17:58:23.121219+0000","last_undegraded":"2023-10-18T17:58:23.121219+0000","last_fullsized":"2023-10-18T17:58:23.121219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.700834+0000","last_change":"2023-10-18T17:58:22.700039+0000","last_active":"2023-10-18T17:58:22.700834+0000","last_peered":"2023-10-18T17:58:22.700834+0000","last_clean":"2023-10-18T17:58:22.700834+0000","last_became_active":"2023-10-18T17:58:22.699719+0000","last_became_peered":"2023-10-18T17:58:22.699719+0000","last_unstale":"2023-10-18T17:58:22.700834+0000","last_undegraded":"2023-10-18T17:58:22.700834+0000","last_fullsized":"2023-10-18T17:58:22.700834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116723+0000","last_change":"2023-10-18T17:58:23.116723+0000","last_active":"2023-10-18T17:58:23.116723+0000","last_peered":"2023-10-18T17:58:23.116723+0000","last_clean":"2023-10-18T17:58:23.116723+0000","last_became_active":"2023-10-18T17:58:23.116376+0000","last_became_peered":"2023-10-18T17:58:23.116376+0000","last_unstale":"2023-10-18T17:58:23.116723+0000","last_undegraded":"2023-10-18T17:58:23.116723+0000","last_fullsized":"2023-10-18T17:58:23.116723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120037+0000","last_change":"2023-10-18T17:58:22.680018+0000","last_active":"2023-10-18T17:58:23.120037+0000","last_peered":"2023-10-18T17:58:23.120037+0000","last_clean":"2023-10-18T17:58:23.120037+0000","last_became_active":"2023-10-18T17:58:22.679464+0000","last_became_peered":"2023-10-18T17:58:22.679464+0000","last_unstale":"2023-10-18T17:58:23.120037+0000","last_undegraded":"2023-10-18T17:58:23.120037+0000","last_fullsized":"2023-10-18T17:58:23.120037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115758+0000","last_change":"2023-10-18T17:58:23.115758+0000","last_active":"2023-10-18T17:58:23.115758+0000","last_peered":"2023-10-18T17:58:23.115758+0000","last_clean":"2023-10-18T17:58:23.115758+0000","last_became_active":"2023-10-18T17:58:23.115471+0000","last_became_peered":"2023-10-18T17:58:23.115471+0000","last_unstale":"2023-10-18T17:58:23.115758+0000","last_undegraded":"2023-10-18T17:58:23.115758+0000","last_fullsized":"2023-10-18T17:58:23.115758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110821+0000","last_change":"2023-10-18T17:58:22.686370+0000","last_active":"2023-10-18T17:58:23.110821+0000","last_peered":"2023-10-18T17:58:23.110821+0000","last_clean":"2023-10-18T17:58:23.110821+0000","last_became_active":"2023-10-18T17:58:22.686062+0000","last_became_peered":"2023-10-18T17:58:22.686062+0000","last_unstale":"2023-10-18T17:58:23.110821+0000","last_undegraded":"2023-10-18T17:58:23.110821+0000","last_fullsized":"2023-10-18T17:58:23.110821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125366+0000","last_change":"2023-10-18T17:58:23.118319+0000","last_active":"2023-10-18T17:58:23.125366+0000","last_peered":"2023-10-18T17:58:23.125366+0000","last_clean":"2023-10-18T17:58:23.125366+0000","last_became_active":"2023-10-18T17:58:23.118018+0000","last_became_peered":"2023-10-18T17:58:23.118018+0000","last_unstale":"2023-10-18T17:58:23.125366+0000","last_undegraded":"2023-10-18T17:58:23.125366+0000","last_fullsized":"2023-10-18T17:58:23.125366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.698386+0000","last_change":"2023-10-18T17:58:22.697114+0000","last_active":"2023-10-18T17:58:22.698386+0000","last_peered":"2023-10-18T17:58:22.698386+0000","last_clean":"2023-10-18T17:58:22.698386+0000","last_became_active":"2023-10-18T17:58:22.696798+0000","last_became_peered":"2023-10-18T17:58:22.696798+0000","last_unstale":"2023-10-18T17:58:22.698386+0000","last_undegraded":"2023-10-18T17:58:22.698386+0000","last_fullsized":"2023-10-18T17:58:22.698386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125558+0000","last_change":"2023-10-18T17:58:23.117398+0000","last_active":"2023-10-18T17:58:23.125558+0000","last_peered":"2023-10-18T17:58:23.125558+0000","last_clean":"2023-10-18T17:58:23.125558+0000","last_became_active":"2023-10-18T17:58:23.117086+0000","last_became_peered":"2023-10-18T17:58:23.117086+0000","last_unstale":"2023-10-18T17:58:23.125558+0000","last_undegraded":"2023-10-18T17:58:23.125558+0000","last_fullsized":"2023-10-18T17:58:23.125558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103780+0000","last_change":"2023-10-18T17:58:22.678915+0000","last_active":"2023-10-18T17:58:23.103780+0000","last_peered":"2023-10-18T17:58:23.103780+0000","last_clean":"2023-10-18T17:58:23.103780+0000","last_became_active":"2023-10-18T17:58:22.678609+0000","last_became_peered":"2023-10-18T17:58:22.678609+0000","last_unstale":"2023-10-18T17:58:23.103780+0000","last_undegraded":"2023-10-18T17:58:23.103780+0000","last_fullsized":"2023-10-18T17:58:23.103780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.122008+0000","last_change":"2023-10-18T17:58:23.122008+0000","last_active":"2023-10-18T17:58:23.122008+0000","last_peered":"2023-10-18T17:58:23.122008+0000","last_clean":"2023-10-18T17:58:23.122008+0000","last_became_active":"2023-10-18T17:58:23.121684+0000","last_became_peered":"2023-10-18T17:58:23.121684+0000","last_unstale":"2023-10-18T17:58:23.122008+0000","last_undegraded":"2023-10-18T17:58:23.122008+0000","last_fullsized":"2023-10-18T17:58:23.122008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.673345+0000","last_change":"2023-10-18T17:58:22.668404+0000","last_active":"2023-10-18T17:58:22.673345+0000","last_peered":"2023-10-18T17:58:22.673345+0000","last_clean":"2023-10-18T17:58:22.673345+0000","last_became_active":"2023-10-18T17:58:22.668038+0000","last_became_peered":"2023-10-18T17:58:22.668038+0000","last_unstale":"2023-10-18T17:58:22.673345+0000","last_undegraded":"2023-10-18T17:58:22.673345+0000","last_fullsized":"2023-10-18T17:58:22.673345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127151+0000","last_change":"2023-10-18T17:58:23.121199+0000","last_active":"2023-10-18T17:58:23.127151+0000","last_peered":"2023-10-18T17:58:23.127151+0000","last_clean":"2023-10-18T17:58:23.127151+0000","last_became_active":"2023-10-18T17:58:23.120856+0000","last_became_peered":"2023-10-18T17:58:23.120856+0000","last_unstale":"2023-10-18T17:58:23.127151+0000","last_undegraded":"2023-10-18T17:58:23.127151+0000","last_fullsized":"2023-10-18T17:58:23.127151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114412+0000","last_change":"2023-10-18T17:58:22.688492+0000","last_active":"2023-10-18T17:58:23.114412+0000","last_peered":"2023-10-18T17:58:23.114412+0000","last_clean":"2023-10-18T17:58:23.114412+0000","last_became_active":"2023-10-18T17:58:22.688162+0000","last_became_peered":"2023-10-18T17:58:22.688162+0000","last_unstale":"2023-10-18T17:58:23.114412+0000","last_undegraded":"2023-10-18T17:58:23.114412+0000","last_fullsized":"2023-10-18T17:58:23.114412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125037+0000","last_change":"2023-10-18T17:58:23.125037+0000","last_active":"2023-10-18T17:58:23.125037+0000","last_peered":"2023-10-18T17:58:23.125037+0000","last_clean":"2023-10-18T17:58:23.125037+0000","last_became_active":"2023-10-18T17:58:23.124722+0000","last_became_peered":"2023-10-18T17:58:23.124722+0000","last_unstale":"2023-10-18T17:58:23.125037+0000","last_undegraded":"2023-10-18T17:58:23.125037+0000","last_fullsized":"2023-10-18T17:58:23.125037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.682504+0000","last_change":"2023-10-18T17:58:22.680359+0000","last_active":"2023-10-18T17:58:22.682504+0000","last_peered":"2023-10-18T17:58:22.682504+0000","last_clean":"2023-10-18T17:58:22.682504+0000","last_became_active":"2023-10-18T17:58:22.679356+0000","last_became_peered":"2023-10-18T17:58:22.679356+0000","last_unstale":"2023-10-18T17:58:22.682504+0000","last_undegraded":"2023-10-18T17:58:22.682504+0000","last_fullsized":"2023-10-18T17:58:22.682504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129449+0000","last_change":"2023-10-18T17:58:23.125443+0000","last_active":"2023-10-18T17:58:23.129449+0000","last_peered":"2023-10-18T17:58:23.129449+0000","last_clean":"2023-10-18T17:58:23.129449+0000","last_became_active":"2023-10-18T17:58:23.124036+0000","last_became_peered":"2023-10-18T17:58:23.124036+0000","last_unstale":"2023-10-18T17:58:23.129449+0000","last_undegraded":"2023-10-18T17:58:23.129449+0000","last_fullsized":"2023-10-18T17:58:23.129449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116673+0000","last_change":"2023-10-18T17:58:22.695695+0000","last_active":"2023-10-18T17:58:23.116673+0000","last_peered":"2023-10-18T17:58:23.116673+0000","last_clean":"2023-10-18T17:58:23.116673+0000","last_became_active":"2023-10-18T17:58:22.694956+0000","last_became_peered":"2023-10-18T17:58:22.694956+0000","last_unstale":"2023-10-18T17:58:23.116673+0000","last_undegraded":"2023-10-18T17:58:23.116673+0000","last_fullsized":"2023-10-18T17:58:23.116673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.124170+0000","last_change":"2023-10-18T17:58:23.124170+0000","last_active":"2023-10-18T17:58:23.124170+0000","last_peered":"2023-10-18T17:58:23.124170+0000","last_clean":"2023-10-18T17:58:23.124170+0000","last_became_active":"2023-10-18T17:58:23.123835+0000","last_became_peered":"2023-10-18T17:58:23.123835+0000","last_unstale":"2023-10-18T17:58:23.124170+0000","last_undegraded":"2023-10-18T17:58:23.124170+0000","last_fullsized":"2023-10-18T17:58:23.124170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683697+0000","last_change":"2023-10-18T17:58:22.678817+0000","last_active":"2023-10-18T17:58:22.683697+0000","last_peered":"2023-10-18T17:58:22.683697+0000","last_clean":"2023-10-18T17:58:22.683697+0000","last_became_active":"2023-10-18T17:58:22.678508+0000","last_became_peered":"2023-10-18T17:58:22.678508+0000","last_unstale":"2023-10-18T17:58:22.683697+0000","last_undegraded":"2023-10-18T17:58:22.683697+0000","last_fullsized":"2023-10-18T17:58:22.683697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.133265+0000","last_change":"2023-10-18T17:58:23.129445+0000","last_active":"2023-10-18T17:58:23.133265+0000","last_peered":"2023-10-18T17:58:23.133265+0000","last_clean":"2023-10-18T17:58:23.133265+0000","last_became_active":"2023-10-18T17:58:23.129038+0000","last_became_peered":"2023-10-18T17:58:23.129038+0000","last_unstale":"2023-10-18T17:58:23.133265+0000","last_undegraded":"2023-10-18T17:58:23.133265+0000","last_fullsized":"2023-10-18T17:58:23.133265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.685131+0000","last_change":"2023-10-18T17:58:22.684469+0000","last_active":"2023-10-18T17:58:22.685131+0000","last_peered":"2023-10-18T17:58:22.685131+0000","last_clean":"2023-10-18T17:58:22.685131+0000","last_became_active":"2023-10-18T17:58:22.684160+0000","last_became_peered":"2023-10-18T17:58:22.684160+0000","last_unstale":"2023-10-18T17:58:22.685131+0000","last_undegraded":"2023-10-18T17:58:22.685131+0000","last_fullsized":"2023-10-18T17:58:22.685131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120664+0000","last_change":"2023-10-18T17:58:23.120664+0000","last_active":"2023-10-18T17:58:23.120664+0000","last_peered":"2023-10-18T17:58:23.120664+0000","last_clean":"2023-10-18T17:58:23.120664+0000","last_became_active":"2023-10-18T17:58:23.120359+0000","last_became_peered":"2023-10-18T17:58:23.120359+0000","last_unstale":"2023-10-18T17:58:23.120664+0000","last_undegraded":"2023-10-18T17:58:23.120664+0000","last_fullsized":"2023-10-18T17:58:23.120664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.680709+0000","last_change":"2023-10-18T17:58:22.676645+0000","last_active":"2023-10-18T17:58:22.680709+0000","last_peered":"2023-10-18T17:58:22.680709+0000","last_clean":"2023-10-18T17:58:22.680709+0000","last_became_active":"2023-10-18T17:58:22.676295+0000","last_became_peered":"2023-10-18T17:58:22.676295+0000","last_unstale":"2023-10-18T17:58:22.680709+0000","last_undegraded":"2023-10-18T17:58:22.680709+0000","last_fullsized":"2023-10-18T17:58:22.680709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.124501+0000","last_change":"2023-10-18T17:58:23.124501+0000","last_active":"2023-10-18T17:58:23.124501+0000","last_peered":"2023-10-18T17:58:23.124501+0000","last_clean":"2023-10-18T17:58:23.124501+0000","last_became_active":"2023-10-18T17:58:23.124186+0000","last_became_peered":"2023-10-18T17:58:23.124186+0000","last_unstale":"2023-10-18T17:58:23.124501+0000","last_undegraded":"2023-10-18T17:58:23.124501+0000","last_fullsized":"2023-10-18T17:58:23.124501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117130+0000","last_change":"2023-10-18T17:58:22.690272+0000","last_active":"2023-10-18T17:58:23.117130+0000","last_peered":"2023-10-18T17:58:23.117130+0000","last_clean":"2023-10-18T17:58:23.117130+0000","last_became_active":"2023-10-18T17:58:22.689949+0000","last_became_peered":"2023-10-18T17:58:22.689949+0000","last_unstale":"2023-10-18T17:58:23.117130+0000","last_undegraded":"2023-10-18T17:58:23.117130+0000","last_fullsized":"2023-10-18T17:58:23.117130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134179+0000","last_change":"2023-10-18T17:58:23.126544+0000","last_active":"2023-10-18T17:58:23.134179+0000","last_peered":"2023-10-18T17:58:23.134179+0000","last_clean":"2023-10-18T17:58:23.134179+0000","last_became_active":"2023-10-18T17:58:23.125987+0000","last_became_peered":"2023-10-18T17:58:23.125987+0000","last_unstale":"2023-10-18T17:58:23.134179+0000","last_undegraded":"2023-10-18T17:58:23.134179+0000","last_fullsized":"2023-10-18T17:58:23.134179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.682755+0000","last_change":"2023-10-18T17:58:22.681429+0000","last_active":"2023-10-18T17:58:22.682755+0000","last_peered":"2023-10-18T17:58:22.682755+0000","last_clean":"2023-10-18T17:58:22.682755+0000","last_became_active":"2023-10-18T17:58:22.681145+0000","last_became_peered":"2023-10-18T17:58:22.681145+0000","last_unstale":"2023-10-18T17:58:22.682755+0000","last_undegraded":"2023-10-18T17:58:22.682755+0000","last_fullsized":"2023-10-18T17:58:22.682755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117504+0000","last_change":"2023-10-18T17:58:23.117504+0000","last_active":"2023-10-18T17:58:23.117504+0000","last_peered":"2023-10-18T17:58:23.117504+0000","last_clean":"2023-10-18T17:58:23.117504+0000","last_became_active":"2023-10-18T17:58:23.117192+0000","last_became_peered":"2023-10-18T17:58:23.117192+0000","last_unstale":"2023-10-18T17:58:23.117504+0000","last_undegraded":"2023-10-18T17:58:23.117504+0000","last_fullsized":"2023-10-18T17:58:23.117504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114324+0000","last_change":"2023-10-18T17:58:22.682889+0000","last_active":"2023-10-18T17:58:23.114324+0000","last_peered":"2023-10-18T17:58:23.114324+0000","last_clean":"2023-10-18T17:58:23.114324+0000","last_became_active":"2023-10-18T17:58:22.682273+0000","last_became_peered":"2023-10-18T17:58:22.682273+0000","last_unstale":"2023-10-18T17:58:23.114324+0000","last_undegraded":"2023-10-18T17:58:23.114324+0000","last_fullsized":"2023-10-18T17:58:23.114324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126856+0000","last_change":"2023-10-18T17:58:23.126856+0000","last_active":"2023-10-18T17:58:23.126856+0000","last_peered":"2023-10-18T17:58:23.126856+0000","last_clean":"2023-10-18T17:58:23.126856+0000","last_became_active":"2023-10-18T17:58:23.126582+0000","last_became_peered":"2023-10-18T17:58:23.126582+0000","last_unstale":"2023-10-18T17:58:23.126856+0000","last_undegraded":"2023-10-18T17:58:23.126856+0000","last_fullsized":"2023-10-18T17:58:23.126856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.684036+0000","last_change":"2023-10-18T17:58:22.680971+0000","last_active":"2023-10-18T17:58:22.684036+0000","last_peered":"2023-10-18T17:58:22.684036+0000","last_clean":"2023-10-18T17:58:22.684036+0000","last_became_active":"2023-10-18T17:58:22.678750+0000","last_became_peered":"2023-10-18T17:58:22.678750+0000","last_unstale":"2023-10-18T17:58:22.684036+0000","last_undegraded":"2023-10-18T17:58:22.684036+0000","last_fullsized":"2023-10-18T17:58:22.684036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135050+0000","last_change":"2023-10-18T17:58:23.130754+0000","last_active":"2023-10-18T17:58:23.135050+0000","last_peered":"2023-10-18T17:58:23.135050+0000","last_clean":"2023-10-18T17:58:23.135050+0000","last_became_active":"2023-10-18T17:58:23.130419+0000","last_became_peered":"2023-10-18T17:58:23.130419+0000","last_unstale":"2023-10-18T17:58:23.135050+0000","last_undegraded":"2023-10-18T17:58:23.135050+0000","last_fullsized":"2023-10-18T17:58:23.135050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.684724+0000","last_change":"2023-10-18T17:58:22.683291+0000","last_active":"2023-10-18T17:58:22.684724+0000","last_peered":"2023-10-18T17:58:22.684724+0000","last_clean":"2023-10-18T17:58:22.684724+0000","last_became_active":"2023-10-18T17:58:22.682926+0000","last_became_peered":"2023-10-18T17:58:22.682926+0000","last_unstale":"2023-10-18T17:58:22.684724+0000","last_undegraded":"2023-10-18T17:58:22.684724+0000","last_fullsized":"2023-10-18T17:58:22.684724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126426+0000","last_change":"2023-10-18T17:58:23.126426+0000","last_active":"2023-10-18T17:58:23.126426+0000","last_peered":"2023-10-18T17:58:23.126426+0000","last_clean":"2023-10-18T17:58:23.126426+0000","last_became_active":"2023-10-18T17:58:23.126114+0000","last_became_peered":"2023-10-18T17:58:23.126114+0000","last_unstale":"2023-10-18T17:58:23.126426+0000","last_undegraded":"2023-10-18T17:58:23.126426+0000","last_fullsized":"2023-10-18T17:58:23.126426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119826+0000","last_change":"2023-10-18T17:58:22.690514+0000","last_active":"2023-10-18T17:58:23.119826+0000","last_peered":"2023-10-18T17:58:23.119826+0000","last_clean":"2023-10-18T17:58:23.119826+0000","last_became_active":"2023-10-18T17:58:22.689427+0000","last_became_peered":"2023-10-18T17:58:22.689427+0000","last_unstale":"2023-10-18T17:58:23.119826+0000","last_undegraded":"2023-10-18T17:58:23.119826+0000","last_fullsized":"2023-10-18T17:58:23.119826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135075+0000","last_change":"2023-10-18T17:58:23.128307+0000","last_active":"2023-10-18T17:58:23.135075+0000","last_peered":"2023-10-18T17:58:23.135075+0000","last_clean":"2023-10-18T17:58:23.135075+0000","last_became_active":"2023-10-18T17:58:23.127998+0000","last_became_peered":"2023-10-18T17:58:23.127998+0000","last_unstale":"2023-10-18T17:58:23.135075+0000","last_undegraded":"2023-10-18T17:58:23.135075+0000","last_fullsized":"2023-10-18T17:58:23.135075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104658+0000","last_change":"2023-10-18T17:58:22.686163+0000","last_active":"2023-10-18T17:58:23.104658+0000","last_peered":"2023-10-18T17:58:23.104658+0000","last_clean":"2023-10-18T17:58:23.104658+0000","last_became_active":"2023-10-18T17:58:22.685950+0000","last_became_peered":"2023-10-18T17:58:22.685950+0000","last_unstale":"2023-10-18T17:58:23.104658+0000","last_undegraded":"2023-10-18T17:58:23.104658+0000","last_fullsized":"2023-10-18T17:58:23.104658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125096+0000","last_change":"2023-10-18T17:58:23.123007+0000","last_active":"2023-10-18T17:58:23.125096+0000","last_peered":"2023-10-18T17:58:23.125096+0000","last_clean":"2023-10-18T17:58:23.125096+0000","last_became_active":"2023-10-18T17:58:23.122656+0000","last_became_peered":"2023-10-18T17:58:23.122656+0000","last_unstale":"2023-10-18T17:58:23.125096+0000","last_undegraded":"2023-10-18T17:58:23.125096+0000","last_fullsized":"2023-10-18T17:58:23.125096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.691067+0000","last_change":"2023-10-18T17:58:22.690142+0000","last_active":"2023-10-18T17:58:22.691067+0000","last_peered":"2023-10-18T17:58:22.691067+0000","last_clean":"2023-10-18T17:58:22.691067+0000","last_became_active":"2023-10-18T17:58:22.689829+0000","last_became_peered":"2023-10-18T17:58:22.689829+0000","last_unstale":"2023-10-18T17:58:22.691067+0000","last_undegraded":"2023-10-18T17:58:22.691067+0000","last_fullsized":"2023-10-18T17:58:22.691067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126572+0000","last_change":"2023-10-18T17:58:23.126572+0000","last_active":"2023-10-18T17:58:23.126572+0000","last_peered":"2023-10-18T17:58:23.126572+0000","last_clean":"2023-10-18T17:58:23.126572+0000","last_became_active":"2023-10-18T17:58:23.125662+0000","last_became_peered":"2023-10-18T17:58:23.125662+0000","last_unstale":"2023-10-18T17:58:23.126572+0000","last_undegraded":"2023-10-18T17:58:23.126572+0000","last_fullsized":"2023-10-18T17:58:23.126572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103952+0000","last_change":"2023-10-18T17:58:22.674168+0000","last_active":"2023-10-18T17:58:23.103952+0000","last_peered":"2023-10-18T17:58:23.103952+0000","last_clean":"2023-10-18T17:58:23.103952+0000","last_became_active":"2023-10-18T17:58:22.668382+0000","last_became_peered":"2023-10-18T17:58:22.668382+0000","last_unstale":"2023-10-18T17:58:23.103952+0000","last_undegraded":"2023-10-18T17:58:23.103952+0000","last_fullsized":"2023-10-18T17:58:23.103952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118550+0000","last_change":"2023-10-18T17:58:23.118550+0000","last_active":"2023-10-18T17:58:23.118550+0000","last_peered":"2023-10-18T17:58:23.118550+0000","last_clean":"2023-10-18T17:58:23.118550+0000","last_became_active":"2023-10-18T17:58:23.118162+0000","last_became_peered":"2023-10-18T17:58:23.118162+0000","last_unstale":"2023-10-18T17:58:23.118550+0000","last_undegraded":"2023-10-18T17:58:23.118550+0000","last_fullsized":"2023-10-18T17:58:23.118550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681225+0000","last_change":"2023-10-18T17:58:22.678474+0000","last_active":"2023-10-18T17:58:22.681225+0000","last_peered":"2023-10-18T17:58:22.681225+0000","last_clean":"2023-10-18T17:58:22.681225+0000","last_became_active":"2023-10-18T17:58:22.678197+0000","last_became_peered":"2023-10-18T17:58:22.678197+0000","last_unstale":"2023-10-18T17:58:22.681225+0000","last_undegraded":"2023-10-18T17:58:22.681225+0000","last_fullsized":"2023-10-18T17:58:22.681225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120777+0000","last_change":"2023-10-18T17:58:23.120777+0000","last_active":"2023-10-18T17:58:23.120777+0000","last_peered":"2023-10-18T17:58:23.120777+0000","last_clean":"2023-10-18T17:58:23.120777+0000","last_became_active":"2023-10-18T17:58:23.116893+0000","last_became_peered":"2023-10-18T17:58:23.116893+0000","last_unstale":"2023-10-18T17:58:23.120777+0000","last_undegraded":"2023-10-18T17:58:23.120777+0000","last_fullsized":"2023-10-18T17:58:23.120777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681135+0000","last_change":"2023-10-18T17:58:22.677025+0000","last_active":"2023-10-18T17:58:22.681135+0000","last_peered":"2023-10-18T17:58:22.681135+0000","last_clean":"2023-10-18T17:58:22.681135+0000","last_became_active":"2023-10-18T17:58:22.676536+0000","last_became_peered":"2023-10-18T17:58:22.676536+0000","last_unstale":"2023-10-18T17:58:22.681135+0000","last_undegraded":"2023-10-18T17:58:22.681135+0000","last_fullsized":"2023-10-18T17:58:22.681135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129785+0000","last_change":"2023-10-18T17:58:23.121230+0000","last_active":"2023-10-18T17:58:23.129785+0000","last_peered":"2023-10-18T17:58:23.129785+0000","last_clean":"2023-10-18T17:58:23.129785+0000","last_became_active":"2023-10-18T17:58:23.120977+0000","last_became_peered":"2023-10-18T17:58:23.120977+0000","last_unstale":"2023-10-18T17:58:23.129785+0000","last_undegraded":"2023-10-18T17:58:23.129785+0000","last_fullsized":"2023-10-18T17:58:23.129785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.692413+0000","last_change":"2023-10-18T17:58:22.691513+0000","last_active":"2023-10-18T17:58:22.692413+0000","last_peered":"2023-10-18T17:58:22.692413+0000","last_clean":"2023-10-18T17:58:22.692413+0000","last_became_active":"2023-10-18T17:58:22.691220+0000","last_became_peered":"2023-10-18T17:58:22.691220+0000","last_unstale":"2023-10-18T17:58:22.692413+0000","last_undegraded":"2023-10-18T17:58:22.692413+0000","last_fullsized":"2023-10-18T17:58:22.692413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128454+0000","last_change":"2023-10-18T17:58:23.128454+0000","last_active":"2023-10-18T17:58:23.128454+0000","last_peered":"2023-10-18T17:58:23.128454+0000","last_clean":"2023-10-18T17:58:23.128454+0000","last_became_active":"2023-10-18T17:58:23.128113+0000","last_became_peered":"2023-10-18T17:58:23.128113+0000","last_unstale":"2023-10-18T17:58:23.128454+0000","last_undegraded":"2023-10-18T17:58:23.128454+0000","last_fullsized":"2023-10-18T17:58:23.128454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116763+0000","last_change":"2023-10-18T17:58:22.695883+0000","last_active":"2023-10-18T17:58:23.116763+0000","last_peered":"2023-10-18T17:58:23.116763+0000","last_clean":"2023-10-18T17:58:23.116763+0000","last_became_active":"2023-10-18T17:58:22.695183+0000","last_became_peered":"2023-10-18T17:58:22.695183+0000","last_unstale":"2023-10-18T17:58:23.116763+0000","last_undegraded":"2023-10-18T17:58:23.116763+0000","last_fullsized":"2023-10-18T17:58:23.116763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.123828+0000","last_change":"2023-10-18T17:58:23.123828+0000","last_active":"2023-10-18T17:58:23.123828+0000","last_peered":"2023-10-18T17:58:23.123828+0000","last_clean":"2023-10-18T17:58:23.123828+0000","last_became_active":"2023-10-18T17:58:23.123552+0000","last_became_peered":"2023-10-18T17:58:23.123552+0000","last_unstale":"2023-10-18T17:58:23.123828+0000","last_undegraded":"2023-10-18T17:58:23.123828+0000","last_fullsized":"2023-10-18T17:58:23.123828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111182+0000","last_change":"2023-10-18T17:58:22.685450+0000","last_active":"2023-10-18T17:58:23.111182+0000","last_peered":"2023-10-18T17:58:23.111182+0000","last_clean":"2023-10-18T17:58:23.111182+0000","last_became_active":"2023-10-18T17:58:22.685168+0000","last_became_peered":"2023-10-18T17:58:22.685168+0000","last_unstale":"2023-10-18T17:58:23.111182+0000","last_undegraded":"2023-10-18T17:58:23.111182+0000","last_fullsized":"2023-10-18T17:58:23.111182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114901+0000","last_change":"2023-10-18T17:58:23.114901+0000","last_active":"2023-10-18T17:58:23.114901+0000","last_peered":"2023-10-18T17:58:23.114901+0000","last_clean":"2023-10-18T17:58:23.114901+0000","last_became_active":"2023-10-18T17:58:23.114590+0000","last_became_peered":"2023-10-18T17:58:23.114590+0000","last_unstale":"2023-10-18T17:58:23.114901+0000","last_undegraded":"2023-10-18T17:58:23.114901+0000","last_fullsized":"2023-10-18T17:58:23.114901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112133+0000","last_change":"2023-10-18T17:58:22.692159+0000","last_active":"2023-10-18T17:58:23.112133+0000","last_peered":"2023-10-18T17:58:23.112133+0000","last_clean":"2023-10-18T17:58:23.112133+0000","last_became_active":"2023-10-18T17:58:22.691846+0000","last_became_peered":"2023-10-18T17:58:22.691846+0000","last_unstale":"2023-10-18T17:58:23.112133+0000","last_undegraded":"2023-10-18T17:58:23.112133+0000","last_fullsized":"2023-10-18T17:58:23.112133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136653+0000","last_change":"2023-10-18T17:58:23.134892+0000","last_active":"2023-10-18T17:58:23.136653+0000","last_peered":"2023-10-18T17:58:23.136653+0000","last_clean":"2023-10-18T17:58:23.136653+0000","last_became_active":"2023-10-18T17:58:23.134213+0000","last_became_peered":"2023-10-18T17:58:23.134213+0000","last_unstale":"2023-10-18T17:58:23.136653+0000","last_undegraded":"2023-10-18T17:58:23.136653+0000","last_fullsized":"2023-10-18T17:58:23.136653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120693+0000","last_change":"2023-10-18T17:58:17.791475+0000","last_active":"2023-10-18T17:58:23.120693+0000","last_peered":"2023-10-18T17:58:23.120693+0000","last_clean":"2023-10-18T17:58:23.120693+0000","last_became_active":"2023-10-18T17:58:17.790774+0000","last_became_peered":"2023-10-18T17:58:17.790774+0000","last_unstale":"2023-10-18T17:58:23.120693+0000","last_undegraded":"2023-10-18T17:58:23.120693+0000","last_fullsized":"2023-10-18T17:58:23.120693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681177+0000","last_change":"2023-10-18T17:58:22.676456+0000","last_active":"2023-10-18T17:58:22.681177+0000","last_peered":"2023-10-18T17:58:22.681177+0000","last_clean":"2023-10-18T17:58:22.681177+0000","last_became_active":"2023-10-18T17:58:22.676189+0000","last_became_peered":"2023-10-18T17:58:22.676189+0000","last_unstale":"2023-10-18T17:58:22.681177+0000","last_undegraded":"2023-10-18T17:58:22.681177+0000","last_fullsized":"2023-10-18T17:58:22.681177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136521+0000","last_change":"2023-10-18T17:58:23.133571+0000","last_active":"2023-10-18T17:58:23.136521+0000","last_peered":"2023-10-18T17:58:23.136521+0000","last_clean":"2023-10-18T17:58:23.136521+0000","last_became_active":"2023-10-18T17:58:23.133302+0000","last_became_peered":"2023-10-18T17:58:23.133302+0000","last_unstale":"2023-10-18T17:58:23.136521+0000","last_undegraded":"2023-10-18T17:58:23.136521+0000","last_fullsized":"2023-10-18T17:58:23.136521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.665338+0000","last_change":"2023-10-18T17:58:17.792368+0000","last_active":"2023-10-18T17:58:22.665338+0000","last_peered":"2023-10-18T17:58:22.665338+0000","last_clean":"2023-10-18T17:58:22.665338+0000","last_became_active":"2023-10-18T17:58:17.791751+0000","last_became_peered":"2023-10-18T17:58:17.791751+0000","last_unstale":"2023-10-18T17:58:22.665338+0000","last_undegraded":"2023-10-18T17:58:22.665338+0000","last_fullsized":"2023-10-18T17:58:22.665338+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.679239+0000","last_change":"2023-10-18T17:58:22.675348+0000","last_active":"2023-10-18T17:58:22.679239+0000","last_peered":"2023-10-18T17:58:22.679239+0000","last_clean":"2023-10-18T17:58:22.679239+0000","last_became_active":"2023-10-18T17:58:22.674469+0000","last_became_peered":"2023-10-18T17:58:22.674469+0000","last_unstale":"2023-10-18T17:58:22.679239+0000","last_undegraded":"2023-10-18T17:58:22.679239+0000","last_fullsized":"2023-10-18T17:58:22.679239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121217+0000","last_change":"2023-10-18T17:58:23.121217+0000","last_active":"2023-10-18T17:58:23.121217+0000","last_peered":"2023-10-18T17:58:23.121217+0000","last_clean":"2023-10-18T17:58:23.121217+0000","last_became_active":"2023-10-18T17:58:23.120691+0000","last_became_peered":"2023-10-18T17:58:23.120691+0000","last_unstale":"2023-10-18T17:58:23.121217+0000","last_undegraded":"2023-10-18T17:58:23.121217+0000","last_fullsized":"2023-10-18T17:58:23.121217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120107+0000","last_change":"2023-10-18T17:58:17.605810+0000","last_active":"2023-10-18T17:58:23.120107+0000","last_peered":"2023-10-18T17:58:23.120107+0000","last_clean":"2023-10-18T17:58:23.120107+0000","last_became_active":"2023-10-18T17:58:17.605289+0000","last_became_peered":"2023-10-18T17:58:17.605289+0000","last_unstale":"2023-10-18T17:58:23.120107+0000","last_undegraded":"2023-10-18T17:58:23.120107+0000","last_fullsized":"2023-10-18T17:58:23.120107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681523+0000","last_change":"2023-10-18T17:58:22.675274+0000","last_active":"2023-10-18T17:58:22.681523+0000","last_peered":"2023-10-18T17:58:22.681523+0000","last_clean":"2023-10-18T17:58:22.681523+0000","last_became_active":"2023-10-18T17:58:22.674965+0000","last_became_peered":"2023-10-18T17:58:22.674965+0000","last_unstale":"2023-10-18T17:58:22.681523+0000","last_undegraded":"2023-10-18T17:58:22.681523+0000","last_fullsized":"2023-10-18T17:58:22.681523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125547+0000","last_change":"2023-10-18T17:58:23.125547+0000","last_active":"2023-10-18T17:58:23.125547+0000","last_peered":"2023-10-18T17:58:23.125547+0000","last_clean":"2023-10-18T17:58:23.125547+0000","last_became_active":"2023-10-18T17:58:23.125259+0000","last_became_peered":"2023-10-18T17:58:23.125259+0000","last_unstale":"2023-10-18T17:58:23.125547+0000","last_undegraded":"2023-10-18T17:58:23.125547+0000","last_fullsized":"2023-10-18T17:58:23.125547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.662363+0000","last_change":"2023-10-18T17:58:18.326048+0000","last_active":"2023-10-18T17:58:22.662363+0000","last_peered":"2023-10-18T17:58:22.662363+0000","last_clean":"2023-10-18T17:58:22.662363+0000","last_became_active":"2023-10-18T17:58:18.325141+0000","last_became_peered":"2023-10-18T17:58:18.325141+0000","last_unstale":"2023-10-18T17:58:22.662363+0000","last_undegraded":"2023-10-18T17:58:22.662363+0000","last_fullsized":"2023-10-18T17:58:22.662363+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114331+0000","last_change":"2023-10-18T17:58:22.684816+0000","last_active":"2023-10-18T17:58:23.114331+0000","last_peered":"2023-10-18T17:58:23.114331+0000","last_clean":"2023-10-18T17:58:23.114331+0000","last_became_active":"2023-10-18T17:58:22.682458+0000","last_became_peered":"2023-10-18T17:58:22.682458+0000","last_unstale":"2023-10-18T17:58:23.114331+0000","last_undegraded":"2023-10-18T17:58:23.114331+0000","last_fullsized":"2023-10-18T17:58:23.114331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.131050+0000","last_change":"2023-10-18T17:58:23.131050+0000","last_active":"2023-10-18T17:58:23.131050+0000","last_peered":"2023-10-18T17:58:23.131050+0000","last_clean":"2023-10-18T17:58:23.131050+0000","last_became_active":"2023-10-18T17:58:23.130768+0000","last_became_peered":"2023-10-18T17:58:23.130768+0000","last_unstale":"2023-10-18T17:58:23.131050+0000","last_undegraded":"2023-10-18T17:58:23.131050+0000","last_fullsized":"2023-10-18T17:58:23.131050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110585+0000","last_change":"2023-10-18T17:58:22.698814+0000","last_active":"2023-10-18T17:58:23.110585+0000","last_peered":"2023-10-18T17:58:23.110585+0000","last_clean":"2023-10-18T17:58:23.110585+0000","last_became_active":"2023-10-18T17:58:22.698525+0000","last_became_peered":"2023-10-18T17:58:22.698525+0000","last_unstale":"2023-10-18T17:58:23.110585+0000","last_undegraded":"2023-10-18T17:58:23.110585+0000","last_fullsized":"2023-10-18T17:58:23.110585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117324+0000","last_change":"2023-10-18T17:58:18.322815+0000","last_active":"2023-10-18T17:58:23.117324+0000","last_peered":"2023-10-18T17:58:23.117324+0000","last_clean":"2023-10-18T17:58:23.117324+0000","last_became_active":"2023-10-18T17:58:18.322057+0000","last_became_peered":"2023-10-18T17:58:18.322057+0000","last_unstale":"2023-10-18T17:58:23.117324+0000","last_undegraded":"2023-10-18T17:58:23.117324+0000","last_fullsized":"2023-10-18T17:58:23.117324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125081+0000","last_change":"2023-10-18T17:58:23.125081+0000","last_active":"2023-10-18T17:58:23.125081+0000","last_peered":"2023-10-18T17:58:23.125081+0000","last_clean":"2023-10-18T17:58:23.125081+0000","last_became_active":"2023-10-18T17:58:23.124833+0000","last_became_peered":"2023-10-18T17:58:23.124833+0000","last_unstale":"2023-10-18T17:58:23.125081+0000","last_undegraded":"2023-10-18T17:58:23.125081+0000","last_fullsized":"2023-10-18T17:58:23.125081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683859+0000","last_change":"2023-10-18T17:58:22.680737+0000","last_active":"2023-10-18T17:58:22.683859+0000","last_peered":"2023-10-18T17:58:22.683859+0000","last_clean":"2023-10-18T17:58:22.683859+0000","last_became_active":"2023-10-18T17:58:22.676981+0000","last_became_peered":"2023-10-18T17:58:22.676981+0000","last_unstale":"2023-10-18T17:58:22.683859+0000","last_undegraded":"2023-10-18T17:58:22.683859+0000","last_fullsized":"2023-10-18T17:58:22.683859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.669846+0000","last_change":"2023-10-18T17:58:17.608107+0000","last_active":"2023-10-18T17:58:22.669846+0000","last_peered":"2023-10-18T17:58:22.669846+0000","last_clean":"2023-10-18T17:58:22.669846+0000","last_became_active":"2023-10-18T17:58:17.607603+0000","last_became_peered":"2023-10-18T17:58:17.607603+0000","last_unstale":"2023-10-18T17:58:22.669846+0000","last_undegraded":"2023-10-18T17:58:22.669846+0000","last_fullsized":"2023-10-18T17:58:22.669846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125950+0000","last_change":"2023-10-18T17:58:23.125950+0000","last_active":"2023-10-18T17:58:23.125950+0000","last_peered":"2023-10-18T17:58:23.125950+0000","last_clean":"2023-10-18T17:58:23.125950+0000","last_became_active":"2023-10-18T17:58:23.125172+0000","last_became_peered":"2023-10-18T17:58:23.125172+0000","last_unstale":"2023-10-18T17:58:23.125950+0000","last_undegraded":"2023-10-18T17:58:23.125950+0000","last_fullsized":"2023-10-18T17:58:23.125950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.684584+0000","last_change":"2023-10-18T17:58:22.683261+0000","last_active":"2023-10-18T17:58:22.684584+0000","last_peered":"2023-10-18T17:58:22.684584+0000","last_clean":"2023-10-18T17:58:22.684584+0000","last_became_active":"2023-10-18T17:58:22.682927+0000","last_became_peered":"2023-10-18T17:58:22.682927+0000","last_unstale":"2023-10-18T17:58:22.684584+0000","last_undegraded":"2023-10-18T17:58:22.684584+0000","last_fullsized":"2023-10-18T17:58:22.684584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114484+0000","last_change":"2023-10-18T17:58:17.607313+0000","last_active":"2023-10-18T17:58:23.114484+0000","last_peered":"2023-10-18T17:58:23.114484+0000","last_clean":"2023-10-18T17:58:23.114484+0000","last_became_active":"2023-10-18T17:58:17.606933+0000","last_became_peered":"2023-10-18T17:58:17.606933+0000","last_unstale":"2023-10-18T17:58:23.114484+0000","last_undegraded":"2023-10-18T17:58:23.114484+0000","last_fullsized":"2023-10-18T17:58:23.114484+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116455+0000","last_change":"2023-10-18T17:58:23.116455+0000","last_active":"2023-10-18T17:58:23.116455+0000","last_peered":"2023-10-18T17:58:23.116455+0000","last_clean":"2023-10-18T17:58:23.116455+0000","last_became_active":"2023-10-18T17:58:23.116050+0000","last_became_peered":"2023-10-18T17:58:23.116050+0000","last_unstale":"2023-10-18T17:58:23.116455+0000","last_undegraded":"2023-10-18T17:58:23.116455+0000","last_fullsized":"2023-10-18T17:58:23.116455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.690614+0000","last_change":"2023-10-18T17:58:22.689750+0000","last_active":"2023-10-18T17:58:22.690614+0000","last_peered":"2023-10-18T17:58:22.690614+0000","last_clean":"2023-10-18T17:58:22.690614+0000","last_became_active":"2023-10-18T17:58:22.689474+0000","last_became_peered":"2023-10-18T17:58:22.689474+0000","last_unstale":"2023-10-18T17:58:22.690614+0000","last_undegraded":"2023-10-18T17:58:22.690614+0000","last_fullsized":"2023-10-18T17:58:22.690614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112122+0000","last_change":"2023-10-18T17:58:17.646977+0000","last_active":"2023-10-18T17:58:23.112122+0000","last_peered":"2023-10-18T17:58:23.112122+0000","last_clean":"2023-10-18T17:58:23.112122+0000","last_became_active":"2023-10-18T17:58:17.646568+0000","last_became_peered":"2023-10-18T17:58:17.646568+0000","last_unstale":"2023-10-18T17:58:23.112122+0000","last_undegraded":"2023-10-18T17:58:23.112122+0000","last_fullsized":"2023-10-18T17:58:23.112122+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.101935+0000","last_change":"2023-10-18T17:58:15.581426+0000","last_active":"2023-10-18T17:58:23.101935+0000","last_peered":"2023-10-18T17:58:23.101935+0000","last_clean":"2023-10-18T17:58:23.101935+0000","last_became_active":"2023-10-18T17:58:15.580900+0000","last_became_peered":"2023-10-18T17:58:15.580900+0000","last_unstale":"2023-10-18T17:58:23.101935+0000","last_undegraded":"2023-10-18T17:58:23.101935+0000","last_fullsized":"2023-10-18T17:58:23.101935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:14.550597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:14.550597+0000","last_clean_scrub_stamp":"2023-10-18T17:58:14.550597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135976+0000","last_change":"2023-10-18T17:58:23.130365+0000","last_active":"2023-10-18T17:58:23.135976+0000","last_peered":"2023-10-18T17:58:23.135976+0000","last_clean":"2023-10-18T17:58:23.135976+0000","last_became_active":"2023-10-18T17:58:23.128150+0000","last_became_peered":"2023-10-18T17:58:23.128150+0000","last_unstale":"2023-10-18T17:58:23.135976+0000","last_undegraded":"2023-10-18T17:58:23.135976+0000","last_fullsized":"2023-10-18T17:58:23.135976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681134+0000","last_change":"2023-10-18T17:58:22.675737+0000","last_active":"2023-10-18T17:58:22.681134+0000","last_peered":"2023-10-18T17:58:22.681134+0000","last_clean":"2023-10-18T17:58:22.681134+0000","last_became_active":"2023-10-18T17:58:22.675306+0000","last_became_peered":"2023-10-18T17:58:22.675306+0000","last_unstale":"2023-10-18T17:58:22.681134+0000","last_undegraded":"2023-10-18T17:58:22.681134+0000","last_fullsized":"2023-10-18T17:58:22.681134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115745+0000","last_change":"2023-10-18T17:58:23.115745+0000","last_active":"2023-10-18T17:58:23.115745+0000","last_peered":"2023-10-18T17:58:23.115745+0000","last_clean":"2023-10-18T17:58:23.115745+0000","last_became_active":"2023-10-18T17:58:23.115454+0000","last_became_peered":"2023-10-18T17:58:23.115454+0000","last_unstale":"2023-10-18T17:58:23.115745+0000","last_undegraded":"2023-10-18T17:58:23.115745+0000","last_fullsized":"2023-10-18T17:58:23.115745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.113848+0000","last_change":"2023-10-18T17:58:22.690810+0000","last_active":"2023-10-18T17:58:23.113848+0000","last_peered":"2023-10-18T17:58:23.113848+0000","last_clean":"2023-10-18T17:58:23.113848+0000","last_became_active":"2023-10-18T17:58:22.690506+0000","last_became_peered":"2023-10-18T17:58:22.690506+0000","last_unstale":"2023-10-18T17:58:23.113848+0000","last_undegraded":"2023-10-18T17:58:23.113848+0000","last_fullsized":"2023-10-18T17:58:23.113848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.133051+0000","last_change":"2023-10-18T17:58:23.126643+0000","last_active":"2023-10-18T17:58:23.133051+0000","last_peered":"2023-10-18T17:58:23.133051+0000","last_clean":"2023-10-18T17:58:23.133051+0000","last_became_active":"2023-10-18T17:58:23.126371+0000","last_became_peered":"2023-10-18T17:58:23.126371+0000","last_unstale":"2023-10-18T17:58:23.133051+0000","last_undegraded":"2023-10-18T17:58:23.133051+0000","last_fullsized":"2023-10-18T17:58:23.133051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117250+0000","last_change":"2023-10-18T17:58:22.690893+0000","last_active":"2023-10-18T17:58:23.117250+0000","last_peered":"2023-10-18T17:58:23.117250+0000","last_clean":"2023-10-18T17:58:23.117250+0000","last_became_active":"2023-10-18T17:58:22.689078+0000","last_became_peered":"2023-10-18T17:58:22.689078+0000","last_unstale":"2023-10-18T17:58:23.117250+0000","last_undegraded":"2023-10-18T17:58:23.117250+0000","last_fullsized":"2023-10-18T17:58:23.117250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135993+0000","last_change":"2023-10-18T17:58:23.130578+0000","last_active":"2023-10-18T17:58:23.135993+0000","last_peered":"2023-10-18T17:58:23.135993+0000","last_clean":"2023-10-18T17:58:23.135993+0000","last_became_active":"2023-10-18T17:58:23.130103+0000","last_became_peered":"2023-10-18T17:58:23.130103+0000","last_unstale":"2023-10-18T17:58:23.135993+0000","last_undegraded":"2023-10-18T17:58:23.135993+0000","last_fullsized":"2023-10-18T17:58:23.135993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683951+0000","last_change":"2023-10-18T17:58:22.682353+0000","last_active":"2023-10-18T17:58:22.683951+0000","last_peered":"2023-10-18T17:58:22.683951+0000","last_clean":"2023-10-18T17:58:22.683951+0000","last_became_active":"2023-10-18T17:58:22.682040+0000","last_became_peered":"2023-10-18T17:58:22.682040+0000","last_unstale":"2023-10-18T17:58:22.683951+0000","last_undegraded":"2023-10-18T17:58:22.683951+0000","last_fullsized":"2023-10-18T17:58:22.683951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127708+0000","last_change":"2023-10-18T17:58:23.120218+0000","last_active":"2023-10-18T17:58:23.127708+0000","last_peered":"2023-10-18T17:58:23.127708+0000","last_clean":"2023-10-18T17:58:23.127708+0000","last_became_active":"2023-10-18T17:58:23.119968+0000","last_became_peered":"2023-10-18T17:58:23.119968+0000","last_unstale":"2023-10-18T17:58:23.127708+0000","last_undegraded":"2023-10-18T17:58:23.127708+0000","last_fullsized":"2023-10-18T17:58:23.127708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110949+0000","last_change":"2023-10-18T17:58:22.679610+0000","last_active":"2023-10-18T17:58:23.110949+0000","last_peered":"2023-10-18T17:58:23.110949+0000","last_clean":"2023-10-18T17:58:23.110949+0000","last_became_active":"2023-10-18T17:58:22.678880+0000","last_became_peered":"2023-10-18T17:58:22.678880+0000","last_unstale":"2023-10-18T17:58:23.110949+0000","last_undegraded":"2023-10-18T17:58:23.110949+0000","last_fullsized":"2023-10-18T17:58:23.110949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118035+0000","last_change":"2023-10-18T17:58:23.118035+0000","last_active":"2023-10-18T17:58:23.118035+0000","last_peered":"2023-10-18T17:58:23.118035+0000","last_clean":"2023-10-18T17:58:23.118035+0000","last_became_active":"2023-10-18T17:58:23.117448+0000","last_became_peered":"2023-10-18T17:58:23.117448+0000","last_unstale":"2023-10-18T17:58:23.118035+0000","last_undegraded":"2023-10-18T17:58:23.118035+0000","last_fullsized":"2023-10-18T17:58:23.118035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119936+0000","last_change":"2023-10-18T17:58:22.688810+0000","last_active":"2023-10-18T17:58:23.119936+0000","last_peered":"2023-10-18T17:58:23.119936+0000","last_clean":"2023-10-18T17:58:23.119936+0000","last_became_active":"2023-10-18T17:58:22.688224+0000","last_became_peered":"2023-10-18T17:58:22.688224+0000","last_unstale":"2023-10-18T17:58:23.119936+0000","last_undegraded":"2023-10-18T17:58:23.119936+0000","last_fullsized":"2023-10-18T17:58:23.119936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136556+0000","last_change":"2023-10-18T17:58:23.133896+0000","last_active":"2023-10-18T17:58:23.136556+0000","last_peered":"2023-10-18T17:58:23.136556+0000","last_clean":"2023-10-18T17:58:23.136556+0000","last_became_active":"2023-10-18T17:58:23.133588+0000","last_became_peered":"2023-10-18T17:58:23.133588+0000","last_unstale":"2023-10-18T17:58:23.136556+0000","last_undegraded":"2023-10-18T17:58:23.136556+0000","last_fullsized":"2023-10-18T17:58:23.136556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116851+0000","last_change":"2023-10-18T17:58:22.696491+0000","last_active":"2023-10-18T17:58:23.116851+0000","last_peered":"2023-10-18T17:58:23.116851+0000","last_clean":"2023-10-18T17:58:23.116851+0000","last_became_active":"2023-10-18T17:58:22.696171+0000","last_became_peered":"2023-10-18T17:58:22.696171+0000","last_unstale":"2023-10-18T17:58:23.116851+0000","last_undegraded":"2023-10-18T17:58:23.116851+0000","last_fullsized":"2023-10-18T17:58:23.116851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125405+0000","last_change":"2023-10-18T17:58:23.116692+0000","last_active":"2023-10-18T17:58:23.125405+0000","last_peered":"2023-10-18T17:58:23.125405+0000","last_clean":"2023-10-18T17:58:23.125405+0000","last_became_active":"2023-10-18T17:58:23.116431+0000","last_became_peered":"2023-10-18T17:58:23.116431+0000","last_unstale":"2023-10-18T17:58:23.125405+0000","last_undegraded":"2023-10-18T17:58:23.125405+0000","last_fullsized":"2023-10-18T17:58:23.125405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111957+0000","last_change":"2023-10-18T17:58:22.694825+0000","last_active":"2023-10-18T17:58:23.111957+0000","last_peered":"2023-10-18T17:58:23.111957+0000","last_clean":"2023-10-18T17:58:23.111957+0000","last_became_active":"2023-10-18T17:58:22.694563+0000","last_became_peered":"2023-10-18T17:58:22.694563+0000","last_unstale":"2023-10-18T17:58:23.111957+0000","last_undegraded":"2023-10-18T17:58:23.111957+0000","last_fullsized":"2023-10-18T17:58:23.111957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126937+0000","last_change":"2023-10-18T17:58:23.120769+0000","last_active":"2023-10-18T17:58:23.126937+0000","last_peered":"2023-10-18T17:58:23.126937+0000","last_clean":"2023-10-18T17:58:23.126937+0000","last_became_active":"2023-10-18T17:58:23.120464+0000","last_became_peered":"2023-10-18T17:58:23.120464+0000","last_unstale":"2023-10-18T17:58:23.126937+0000","last_undegraded":"2023-10-18T17:58:23.126937+0000","last_fullsized":"2023-10-18T17:58:23.126937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.101806+0000","last_change":"2023-10-18T17:58:22.668326+0000","last_active":"2023-10-18T17:58:23.101806+0000","last_peered":"2023-10-18T17:58:23.101806+0000","last_clean":"2023-10-18T17:58:23.101806+0000","last_became_active":"2023-10-18T17:58:22.668059+0000","last_became_peered":"2023-10-18T17:58:22.668059+0000","last_unstale":"2023-10-18T17:58:23.101806+0000","last_undegraded":"2023-10-18T17:58:23.101806+0000","last_fullsized":"2023-10-18T17:58:23.101806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120596+0000","last_change":"2023-10-18T17:58:23.120596+0000","last_active":"2023-10-18T17:58:23.120596+0000","last_peered":"2023-10-18T17:58:23.120596+0000","last_clean":"2023-10-18T17:58:23.120596+0000","last_became_active":"2023-10-18T17:58:23.120309+0000","last_became_peered":"2023-10-18T17:58:23.120309+0000","last_unstale":"2023-10-18T17:58:23.120596+0000","last_undegraded":"2023-10-18T17:58:23.120596+0000","last_fullsized":"2023-10-18T17:58:23.120596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.113755+0000","last_change":"2023-10-18T17:58:22.700021+0000","last_active":"2023-10-18T17:58:23.113755+0000","last_peered":"2023-10-18T17:58:23.113755+0000","last_clean":"2023-10-18T17:58:23.113755+0000","last_became_active":"2023-10-18T17:58:22.699570+0000","last_became_peered":"2023-10-18T17:58:22.699570+0000","last_unstale":"2023-10-18T17:58:23.113755+0000","last_undegraded":"2023-10-18T17:58:23.113755+0000","last_fullsized":"2023-10-18T17:58:23.113755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118771+0000","last_change":"2023-10-18T17:58:23.118771+0000","last_active":"2023-10-18T17:58:23.118771+0000","last_peered":"2023-10-18T17:58:23.118771+0000","last_clean":"2023-10-18T17:58:23.118771+0000","last_became_active":"2023-10-18T17:58:23.118493+0000","last_became_peered":"2023-10-18T17:58:23.118493+0000","last_unstale":"2023-10-18T17:58:23.118771+0000","last_undegraded":"2023-10-18T17:58:23.118771+0000","last_fullsized":"2023-10-18T17:58:23.118771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.695181+0000","last_change":"2023-10-18T17:58:22.694576+0000","last_active":"2023-10-18T17:58:22.695181+0000","last_peered":"2023-10-18T17:58:22.695181+0000","last_clean":"2023-10-18T17:58:22.695181+0000","last_became_active":"2023-10-18T17:58:22.694240+0000","last_became_peered":"2023-10-18T17:58:22.694240+0000","last_unstale":"2023-10-18T17:58:22.695181+0000","last_undegraded":"2023-10-18T17:58:22.695181+0000","last_fullsized":"2023-10-18T17:58:22.695181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134198+0000","last_change":"2023-10-18T17:58:23.126726+0000","last_active":"2023-10-18T17:58:23.134198+0000","last_peered":"2023-10-18T17:58:23.134198+0000","last_clean":"2023-10-18T17:58:23.134198+0000","last_became_active":"2023-10-18T17:58:23.126297+0000","last_became_peered":"2023-10-18T17:58:23.126297+0000","last_unstale":"2023-10-18T17:58:23.134198+0000","last_undegraded":"2023-10-18T17:58:23.134198+0000","last_fullsized":"2023-10-18T17:58:23.134198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112044+0000","last_change":"2023-10-18T17:58:22.690048+0000","last_active":"2023-10-18T17:58:23.112044+0000","last_peered":"2023-10-18T17:58:23.112044+0000","last_clean":"2023-10-18T17:58:23.112044+0000","last_became_active":"2023-10-18T17:58:22.689785+0000","last_became_peered":"2023-10-18T17:58:22.689785+0000","last_unstale":"2023-10-18T17:58:23.112044+0000","last_undegraded":"2023-10-18T17:58:23.112044+0000","last_fullsized":"2023-10-18T17:58:23.112044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129919+0000","last_change":"2023-10-18T17:58:23.129919+0000","last_active":"2023-10-18T17:58:23.129919+0000","last_peered":"2023-10-18T17:58:23.129919+0000","last_clean":"2023-10-18T17:58:23.129919+0000","last_became_active":"2023-10-18T17:58:23.129560+0000","last_became_peered":"2023-10-18T17:58:23.129560+0000","last_unstale":"2023-10-18T17:58:23.129919+0000","last_undegraded":"2023-10-18T17:58:23.129919+0000","last_fullsized":"2023-10-18T17:58:23.129919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683706+0000","last_change":"2023-10-18T17:58:22.678801+0000","last_active":"2023-10-18T17:58:22.683706+0000","last_peered":"2023-10-18T17:58:22.683706+0000","last_clean":"2023-10-18T17:58:22.683706+0000","last_became_active":"2023-10-18T17:58:22.678493+0000","last_became_peered":"2023-10-18T17:58:22.678493+0000","last_unstale":"2023-10-18T17:58:22.683706+0000","last_undegraded":"2023-10-18T17:58:22.683706+0000","last_fullsized":"2023-10-18T17:58:22.683706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129374+0000","last_change":"2023-10-18T17:58:23.124984+0000","last_active":"2023-10-18T17:58:23.129374+0000","last_peered":"2023-10-18T17:58:23.129374+0000","last_clean":"2023-10-18T17:58:23.129374+0000","last_became_active":"2023-10-18T17:58:23.124646+0000","last_became_peered":"2023-10-18T17:58:23.124646+0000","last_unstale":"2023-10-18T17:58:23.129374+0000","last_undegraded":"2023-10-18T17:58:23.129374+0000","last_fullsized":"2023-10-18T17:58:23.129374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681640+0000","last_change":"2023-10-18T17:58:22.677885+0000","last_active":"2023-10-18T17:58:22.681640+0000","last_peered":"2023-10-18T17:58:22.681640+0000","last_clean":"2023-10-18T17:58:22.681640+0000","last_became_active":"2023-10-18T17:58:22.677592+0000","last_became_peered":"2023-10-18T17:58:22.677592+0000","last_unstale":"2023-10-18T17:58:22.681640+0000","last_undegraded":"2023-10-18T17:58:22.681640+0000","last_fullsized":"2023-10-18T17:58:22.681640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115290+0000","last_change":"2023-10-18T17:58:23.115290+0000","last_active":"2023-10-18T17:58:23.115290+0000","last_peered":"2023-10-18T17:58:23.115290+0000","last_clean":"2023-10-18T17:58:23.115290+0000","last_became_active":"2023-10-18T17:58:23.115014+0000","last_became_peered":"2023-10-18T17:58:23.115014+0000","last_unstale":"2023-10-18T17:58:23.115290+0000","last_undegraded":"2023-10-18T17:58:23.115290+0000","last_fullsized":"2023-10-18T17:58:23.115290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110719+0000","last_change":"2023-10-18T17:58:22.695980+0000","last_active":"2023-10-18T17:58:23.110719+0000","last_peered":"2023-10-18T17:58:23.110719+0000","last_clean":"2023-10-18T17:58:23.110719+0000","last_became_active":"2023-10-18T17:58:22.695592+0000","last_became_peered":"2023-10-18T17:58:22.695592+0000","last_unstale":"2023-10-18T17:58:23.110719+0000","last_undegraded":"2023-10-18T17:58:23.110719+0000","last_fullsized":"2023-10-18T17:58:23.110719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.124321+0000","last_change":"2023-10-18T17:58:23.124321+0000","last_active":"2023-10-18T17:58:23.124321+0000","last_peered":"2023-10-18T17:58:23.124321+0000","last_clean":"2023-10-18T17:58:23.124321+0000","last_became_active":"2023-10-18T17:58:23.124019+0000","last_became_peered":"2023-10-18T17:58:23.124019+0000","last_unstale":"2023-10-18T17:58:23.124321+0000","last_undegraded":"2023-10-18T17:58:23.124321+0000","last_fullsized":"2023-10-18T17:58:23.124321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111016+0000","last_change":"2023-10-18T17:58:22.692184+0000","last_active":"2023-10-18T17:58:23.111016+0000","last_peered":"2023-10-18T17:58:23.111016+0000","last_clean":"2023-10-18T17:58:23.111016+0000","last_became_active":"2023-10-18T17:58:22.691802+0000","last_became_peered":"2023-10-18T17:58:22.691802+0000","last_unstale":"2023-10-18T17:58:23.111016+0000","last_undegraded":"2023-10-18T17:58:23.111016+0000","last_fullsized":"2023-10-18T17:58:23.111016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126067+0000","last_change":"2023-10-18T17:58:23.116039+0000","last_active":"2023-10-18T17:58:23.126067+0000","last_peered":"2023-10-18T17:58:23.126067+0000","last_clean":"2023-10-18T17:58:23.126067+0000","last_became_active":"2023-10-18T17:58:23.115715+0000","last_became_peered":"2023-10-18T17:58:23.115715+0000","last_unstale":"2023-10-18T17:58:23.126067+0000","last_undegraded":"2023-10-18T17:58:23.126067+0000","last_fullsized":"2023-10-18T17:58:23.126067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111682+0000","last_change":"2023-10-18T17:58:22.687555+0000","last_active":"2023-10-18T17:58:23.111682+0000","last_peered":"2023-10-18T17:58:23.111682+0000","last_clean":"2023-10-18T17:58:23.111682+0000","last_became_active":"2023-10-18T17:58:22.687269+0000","last_became_peered":"2023-10-18T17:58:22.687269+0000","last_unstale":"2023-10-18T17:58:23.111682+0000","last_undegraded":"2023-10-18T17:58:23.111682+0000","last_fullsized":"2023-10-18T17:58:23.111682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127657+0000","last_change":"2023-10-18T17:58:23.120769+0000","last_active":"2023-10-18T17:58:23.127657+0000","last_peered":"2023-10-18T17:58:23.127657+0000","last_clean":"2023-10-18T17:58:23.127657+0000","last_became_active":"2023-10-18T17:58:23.120522+0000","last_became_peered":"2023-10-18T17:58:23.120522+0000","last_unstale":"2023-10-18T17:58:23.127657+0000","last_undegraded":"2023-10-18T17:58:23.127657+0000","last_fullsized":"2023-10-18T17:58:23.127657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104694+0000","last_change":"2023-10-18T17:58:22.689656+0000","last_active":"2023-10-18T17:58:23.104694+0000","last_peered":"2023-10-18T17:58:23.104694+0000","last_clean":"2023-10-18T17:58:23.104694+0000","last_became_active":"2023-10-18T17:58:22.689376+0000","last_became_peered":"2023-10-18T17:58:22.689376+0000","last_unstale":"2023-10-18T17:58:23.104694+0000","last_undegraded":"2023-10-18T17:58:23.104694+0000","last_fullsized":"2023-10-18T17:58:23.104694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126926+0000","last_change":"2023-10-18T17:58:23.120357+0000","last_active":"2023-10-18T17:58:23.126926+0000","last_peered":"2023-10-18T17:58:23.126926+0000","last_clean":"2023-10-18T17:58:23.126926+0000","last_became_active":"2023-10-18T17:58:23.118878+0000","last_became_peered":"2023-10-18T17:58:23.118878+0000","last_unstale":"2023-10-18T17:58:23.126926+0000","last_undegraded":"2023-10-18T17:58:23.126926+0000","last_fullsized":"2023-10-18T17:58:23.126926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126262+0000","last_change":"2023-10-18T17:58:23.126262+0000","last_active":"2023-10-18T17:58:23.126262+0000","last_peered":"2023-10-18T17:58:23.126262+0000","last_clean":"2023-10-18T17:58:23.126262+0000","last_became_active":"2023-10-18T17:58:23.125988+0000","last_became_peered":"2023-10-18T17:58:23.125988+0000","last_unstale":"2023-10-18T17:58:23.126262+0000","last_undegraded":"2023-10-18T17:58:23.126262+0000","last_fullsized":"2023-10-18T17:58:23.126262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683881+0000","last_change":"2023-10-18T17:58:22.680967+0000","last_active":"2023-10-18T17:58:22.683881+0000","last_peered":"2023-10-18T17:58:22.683881+0000","last_clean":"2023-10-18T17:58:22.683881+0000","last_became_active":"2023-10-18T17:58:22.679976+0000","last_became_peered":"2023-10-18T17:58:22.679976+0000","last_unstale":"2023-10-18T17:58:22.683881+0000","last_undegraded":"2023-10-18T17:58:22.683881+0000","last_fullsized":"2023-10-18T17:58:22.683881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135616+0000","last_change":"2023-10-18T17:58:23.131014+0000","last_active":"2023-10-18T17:58:23.135616+0000","last_peered":"2023-10-18T17:58:23.135616+0000","last_clean":"2023-10-18T17:58:23.135616+0000","last_became_active":"2023-10-18T17:58:23.130516+0000","last_became_peered":"2023-10-18T17:58:23.130516+0000","last_unstale":"2023-10-18T17:58:23.135616+0000","last_undegraded":"2023-10-18T17:58:23.135616+0000","last_fullsized":"2023-10-18T17:58:23.135616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.109880+0000","last_change":"2023-10-18T17:58:22.684193+0000","last_active":"2023-10-18T17:58:23.109880+0000","last_peered":"2023-10-18T17:58:23.109880+0000","last_clean":"2023-10-18T17:58:23.109880+0000","last_became_active":"2023-10-18T17:58:22.683668+0000","last_became_peered":"2023-10-18T17:58:22.683668+0000","last_unstale":"2023-10-18T17:58:23.109880+0000","last_undegraded":"2023-10-18T17:58:23.109880+0000","last_fullsized":"2023-10-18T17:58:23.109880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128357+0000","last_change":"2023-10-18T17:58:23.123700+0000","last_active":"2023-10-18T17:58:23.128357+0000","last_peered":"2023-10-18T17:58:23.128357+0000","last_clean":"2023-10-18T17:58:23.128357+0000","last_became_active":"2023-10-18T17:58:23.123224+0000","last_became_peered":"2023-10-18T17:58:23.123224+0000","last_unstale":"2023-10-18T17:58:23.128357+0000","last_undegraded":"2023-10-18T17:58:23.128357+0000","last_fullsized":"2023-10-18T17:58:23.128357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116459+0000","last_change":"2023-10-18T17:58:22.674799+0000","last_active":"2023-10-18T17:58:23.116459+0000","last_peered":"2023-10-18T17:58:23.116459+0000","last_clean":"2023-10-18T17:58:23.116459+0000","last_became_active":"2023-10-18T17:58:22.674479+0000","last_became_peered":"2023-10-18T17:58:22.674479+0000","last_unstale":"2023-10-18T17:58:23.116459+0000","last_undegraded":"2023-10-18T17:58:23.116459+0000","last_fullsized":"2023-10-18T17:58:23.116459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127727+0000","last_change":"2023-10-18T17:58:23.121289+0000","last_active":"2023-10-18T17:58:23.127727+0000","last_peered":"2023-10-18T17:58:23.127727+0000","last_clean":"2023-10-18T17:58:23.127727+0000","last_became_active":"2023-10-18T17:58:23.120024+0000","last_became_peered":"2023-10-18T17:58:23.120024+0000","last_unstale":"2023-10-18T17:58:23.127727+0000","last_undegraded":"2023-10-18T17:58:23.127727+0000","last_fullsized":"2023-10-18T17:58:23.127727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110283+0000","last_change":"2023-10-18T17:58:22.675422+0000","last_active":"2023-10-18T17:58:23.110283+0000","last_peered":"2023-10-18T17:58:23.110283+0000","last_clean":"2023-10-18T17:58:23.110283+0000","last_became_active":"2023-10-18T17:58:22.675074+0000","last_became_peered":"2023-10-18T17:58:22.675074+0000","last_unstale":"2023-10-18T17:58:23.110283+0000","last_undegraded":"2023-10-18T17:58:23.110283+0000","last_fullsized":"2023-10-18T17:58:23.110283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125653+0000","last_change":"2023-10-18T17:58:23.125653+0000","last_active":"2023-10-18T17:58:23.125653+0000","last_peered":"2023-10-18T17:58:23.125653+0000","last_clean":"2023-10-18T17:58:23.125653+0000","last_became_active":"2023-10-18T17:58:23.124682+0000","last_became_peered":"2023-10-18T17:58:23.124682+0000","last_unstale":"2023-10-18T17:58:23.125653+0000","last_undegraded":"2023-10-18T17:58:23.125653+0000","last_fullsized":"2023-10-18T17:58:23.125653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115530+0000","last_change":"2023-10-18T17:58:22.673247+0000","last_active":"2023-10-18T17:58:23.115530+0000","last_peered":"2023-10-18T17:58:23.115530+0000","last_clean":"2023-10-18T17:58:23.115530+0000","last_became_active":"2023-10-18T17:58:22.672903+0000","last_became_peered":"2023-10-18T17:58:22.672903+0000","last_unstale":"2023-10-18T17:58:23.115530+0000","last_undegraded":"2023-10-18T17:58:23.115530+0000","last_fullsized":"2023-10-18T17:58:23.115530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129965+0000","last_change":"2023-10-18T17:58:23.125035+0000","last_active":"2023-10-18T17:58:23.129965+0000","last_peered":"2023-10-18T17:58:23.129965+0000","last_clean":"2023-10-18T17:58:23.129965+0000","last_became_active":"2023-10-18T17:58:23.124761+0000","last_became_peered":"2023-10-18T17:58:23.124761+0000","last_unstale":"2023-10-18T17:58:23.129965+0000","last_undegraded":"2023-10-18T17:58:23.129965+0000","last_fullsized":"2023-10-18T17:58:23.129965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.674667+0000","last_change":"2023-10-18T17:58:22.672712+0000","last_active":"2023-10-18T17:58:22.674667+0000","last_peered":"2023-10-18T17:58:22.674667+0000","last_clean":"2023-10-18T17:58:22.674667+0000","last_became_active":"2023-10-18T17:58:22.672368+0000","last_became_peered":"2023-10-18T17:58:22.672368+0000","last_unstale":"2023-10-18T17:58:22.674667+0000","last_undegraded":"2023-10-18T17:58:22.674667+0000","last_fullsized":"2023-10-18T17:58:22.674667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135798+0000","last_change":"2023-10-18T17:58:23.127848+0000","last_active":"2023-10-18T17:58:23.135798+0000","last_peered":"2023-10-18T17:58:23.135798+0000","last_clean":"2023-10-18T17:58:23.135798+0000","last_became_active":"2023-10-18T17:58:23.125611+0000","last_became_peered":"2023-10-18T17:58:23.125611+0000","last_unstale":"2023-10-18T17:58:23.135798+0000","last_undegraded":"2023-10-18T17:58:23.135798+0000","last_fullsized":"2023-10-18T17:58:23.135798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.679422+0000","last_change":"2023-10-18T17:58:22.675355+0000","last_active":"2023-10-18T17:58:22.679422+0000","last_peered":"2023-10-18T17:58:22.679422+0000","last_clean":"2023-10-18T17:58:22.679422+0000","last_became_active":"2023-10-18T17:58:22.675050+0000","last_became_peered":"2023-10-18T17:58:22.675050+0000","last_unstale":"2023-10-18T17:58:22.679422+0000","last_undegraded":"2023-10-18T17:58:22.679422+0000","last_fullsized":"2023-10-18T17:58:22.679422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129429+0000","last_change":"2023-10-18T17:58:23.124889+0000","last_active":"2023-10-18T17:58:23.129429+0000","last_peered":"2023-10-18T17:58:23.129429+0000","last_clean":"2023-10-18T17:58:23.129429+0000","last_became_active":"2023-10-18T17:58:23.124619+0000","last_became_peered":"2023-10-18T17:58:23.124619+0000","last_unstale":"2023-10-18T17:58:23.129429+0000","last_undegraded":"2023-10-18T17:58:23.129429+0000","last_fullsized":"2023-10-18T17:58:23.129429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111694+0000","last_change":"2023-10-18T17:58:22.692437+0000","last_active":"2023-10-18T17:58:23.111694+0000","last_peered":"2023-10-18T17:58:23.111694+0000","last_clean":"2023-10-18T17:58:23.111694+0000","last_became_active":"2023-10-18T17:58:22.692043+0000","last_became_peered":"2023-10-18T17:58:22.692043+0000","last_unstale":"2023-10-18T17:58:23.111694+0000","last_undegraded":"2023-10-18T17:58:23.111694+0000","last_fullsized":"2023-10-18T17:58:23.111694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7769999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.202}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.881}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.387}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2429999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.601}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297292,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074548,"statfs":{"total":96636764160,"available":96332337152,"internally_reserved":0,"allocated":200704,"data_stored":70752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T17:58:29.381 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T17:58:29.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.570+0000 7f41790db700 1 Processor -- start 2023-10-18T17:58:29.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.570+0000 7f41790db700 1 -- start start 2023-10-18T17:58:29.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f41790db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 0x7f4174067e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f41790db700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41741551c0 con 0x7f41740b55b0 2023-10-18T17:58:29.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f41790db700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41741543a0 con 0x7f4174069d80 2023-10-18T17:58:29.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f41790db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4174154640 con 0x7f41740bc2f0 2023-10-18T17:58:29.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f4172d9d700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f41740b55b0 0x7f41740bcab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50546/0 (socket says 172.21.15.72:50546) 2023-10-18T17:58:29.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f4172d9d700 1 -- 172.21.15.72:0/862895474 learned_addr learned my addr 172.21.15.72:0/862895474 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:29.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f416bfff700 1 --2- 172.21.15.72:0/862895474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 0x7f4174067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f417259c700 1 -- 172.21.15.72:0/862895474 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449178066 0 0) 0x7f41741551c0 con 0x7f41740b55b0 2023-10-18T17:58:29.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f417259c700 1 -- 172.21.15.72:0/862895474 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41600033f0 con 0x7f41740b55b0 2023-10-18T17:58:29.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f417259c700 1 -- 172.21.15.72:0/862895474 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232271996 0 0) 0x7f41741543a0 con 0x7f4174069d80 2023-10-18T17:58:29.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f417259c700 1 -- 172.21.15.72:0/862895474 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41741551c0 con 0x7f4174069d80 2023-10-18T17:58:29.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f416bfff700 1 -- 172.21.15.72:0/862895474 >> v1:172.21.15.179:6789/0 conn(0x7f4174069d80 legacy=0x7f4174065390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f416bfff700 1 -- 172.21.15.72:0/862895474 >> v1:172.21.15.72:6789/0 conn(0x7f41740b55b0 legacy=0x7f41740bcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f416bfff700 1 -- 172.21.15.72:0/862895474 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4174154920 con 0x7f41740bc2f0 2023-10-18T17:58:29.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.574+0000 7f416bfff700 1 --2- 172.21.15.72:0/862895474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 0x7f4174067e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f415c009790 tx=0x7f415c009060).ready entity=mon.2 client_cookie=f1cb4fc46e9a6675 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:29.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f417259c700 1 -- 172.21.15.72:0/862895474 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c00f510 con 0x7f41740bc2f0 2023-10-18T17:58:29.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f417259c700 1 -- 172.21.15.72:0/862895474 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c00f670 con 0x7f41740bc2f0 2023-10-18T17:58:29.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f417259c700 1 -- 172.21.15.72:0/862895474 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c00f9f0 con 0x7f41740bc2f0 2023-10-18T17:58:29.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f41790db700 1 -- 172.21.15.72:0/862895474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 msgr2=0x7f4174067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f41790db700 1 --2- 172.21.15.72:0/862895474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 0x7f4174067e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f415c009790 tx=0x7f415c009060).stop 2023-10-18T17:58:29.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f41790db700 1 -- 172.21.15.72:0/862895474 shutdown_connections 2023-10-18T17:58:29.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f41790db700 1 --2- 172.21.15.72:0/862895474 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 0x7f4174067e30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f41790db700 1 -- 172.21.15.72:0/862895474 >> 172.21.15.72:0/862895474 conn(0x7f41740620d0 msgr2=0x7f41740624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:29.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f41790db700 1 -- 172.21.15.72:0/862895474 shutdown_connections 2023-10-18T17:58:29.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f41790db700 1 -- 172.21.15.72:0/862895474 wait complete. 2023-10-18T17:58:29.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f41790db700 1 Processor -- start 2023-10-18T17:58:29.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.578+0000 7f41790db700 1 -- start start 2023-10-18T17:58:29.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f41790db700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4174069d80 0x7f4174155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f41790db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41740b55b0 0x7f41741681d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f41790db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 0x7f4174157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f41790db700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41740b73f0 con 0x7f4174069d80 2023-10-18T17:58:29.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f41790db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417414e320 con 0x7f41740b55b0 2023-10-18T17:58:29.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f41790db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4174167d40 con 0x7f41740bc2f0 2023-10-18T17:58:29.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f4172d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4174069d80 0x7f4174155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f416bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41740b55b0 0x7f41741681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f4172d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4174069d80 0x7f4174155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58814/0 (socket says 172.21.15.72:58814) 2023-10-18T17:58:29.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f4172d9d700 1 -- 172.21.15.72:0/828048115 learned_addr learned my addr 172.21.15.72:0/828048115 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:29.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f416bfff700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41740b55b0 0x7f41741681d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51228/0 (socket says 172.21.15.72:51228) 2023-10-18T17:58:29.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f417359e700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 0x7f4174157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f4172d9d700 1 -- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 msgr2=0x7f4174157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f4172d9d700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 0x7f4174157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f4172d9d700 1 -- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41740b55b0 msgr2=0x7f41741681d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f4172d9d700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41740b55b0 0x7f41741681d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f4172d9d700 1 -- 172.21.15.72:0/828048115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0092a0 con 0x7f4174069d80 2023-10-18T17:58:29.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f4172d9d700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4174069d80 0x7f4174155810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f416400c4c0 tx=0x7f4164007d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:29.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f416affd700 1 -- 172.21.15.72:0/828048115 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416400ce80 con 0x7f4174069d80 2023-10-18T17:58:29.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f416affd700 1 -- 172.21.15.72:0/828048115 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41640072e0 con 0x7f4174069d80 2023-10-18T17:58:29.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4174153320 con 0x7f4174069d80 2023-10-18T17:58:29.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f416affd700 1 -- 172.21.15.72:0/828048115 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164007660 con 0x7f4174069d80 2023-10-18T17:58:29.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.582+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41740b6280 con 0x7f4174069d80 2023-10-18T17:58:29.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.586+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417406c190 con 0x7f4174069d80 2023-10-18T17:58:29.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.586+0000 7f416affd700 1 -- 172.21.15.72:0/828048115 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f41640059b0 con 0x7f4174069d80 2023-10-18T17:58:29.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.586+0000 7f416affd700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415405a510 0x7f415405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.586+0000 7f416affd700 1 -- 172.21.15.72:0/828048115 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4164073f70 con 0x7f4174069d80 2023-10-18T17:58:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.586+0000 7f416bfff700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415405a510 0x7f415405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:29.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.590+0000 7f416bfff700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415405a510 0x7f415405c9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f415c009fd0 tx=0x7f415c017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:29.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.594+0000 7f416affd700 1 -- 172.21.15.72:0/828048115 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f4164047690 con 0x7f4174069d80 2023-10-18T17:58:29.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.754+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 --> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4174167e80 con 0x7f415405a510 2023-10-18T17:58:29.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.774+0000 7f416affd700 1 -- 172.21.15.72:0/828048115 <== mgr.4102 v2:172.21.15.179:6832/33696 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336124 (secure 0 0 0) 0x7f415c00f7e0 con 0x7f415405a510 2023-10-18T17:58:29.780 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:29.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415405a510 msgr2=0x7f415405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415405a510 0x7f415405c9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f415c009fd0 tx=0x7f415c017000).stop 2023-10-18T17:58:29.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4174069d80 msgr2=0x7f4174155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:29.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4174069d80 0x7f4174155810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f416400c4c0 tx=0x7f4164007d00).stop 2023-10-18T17:58:29.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 shutdown_connections 2023-10-18T17:58:29.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f415405a510 0x7f415405c9d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41740bc2f0 0x7f4174157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41740b55b0 0x7f41741681d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 --2- 172.21.15.72:0/828048115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4174069d80 0x7f4174155810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:29.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 >> 172.21.15.72:0/828048115 conn(0x7f41740620d0 msgr2=0x7f41740bf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:29.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 shutdown_connections 2023-10-18T17:58:29.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:29.782+0000 7f41790db700 1 -- 172.21.15.72:0/828048115 wait complete. 2023-10-18T17:58:29.789 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-10-18T17:58:29.804 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-18T17:58:28.468052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378468,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596252,"statfs":{"total":773094113280,"available":770658562048,"internally_reserved":0,"allocated":1740800,"data_stored":643085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129427+0000","last_change":"2023-10-18T17:58:23.125789+0000","last_active":"2023-10-18T17:58:23.129427+0000","last_peered":"2023-10-18T17:58:23.129427+0000","last_clean":"2023-10-18T17:58:23.129427+0000","last_became_active":"2023-10-18T17:58:23.123952+0000","last_became_peered":"2023-10-18T17:58:23.123952+0000","last_unstale":"2023-10-18T17:58:23.129427+0000","last_undegraded":"2023-10-18T17:58:23.129427+0000","last_fullsized":"2023-10-18T17:58:23.129427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.682906+0000","last_change":"2023-10-18T17:58:22.680518+0000","last_active":"2023-10-18T17:58:22.682906+0000","last_peered":"2023-10-18T17:58:22.682906+0000","last_clean":"2023-10-18T17:58:22.682906+0000","last_became_active":"2023-10-18T17:58:22.679123+0000","last_became_peered":"2023-10-18T17:58:22.679123+0000","last_unstale":"2023-10-18T17:58:22.682906+0000","last_undegraded":"2023-10-18T17:58:22.682906+0000","last_fullsized":"2023-10-18T17:58:22.682906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121498+0000","last_change":"2023-10-18T17:58:23.121498+0000","last_active":"2023-10-18T17:58:23.121498+0000","last_peered":"2023-10-18T17:58:23.121498+0000","last_clean":"2023-10-18T17:58:23.121498+0000","last_became_active":"2023-10-18T17:58:23.121129+0000","last_became_peered":"2023-10-18T17:58:23.121129+0000","last_unstale":"2023-10-18T17:58:23.121498+0000","last_undegraded":"2023-10-18T17:58:23.121498+0000","last_fullsized":"2023-10-18T17:58:23.121498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114330+0000","last_change":"2023-10-18T17:58:22.701312+0000","last_active":"2023-10-18T17:58:23.114330+0000","last_peered":"2023-10-18T17:58:23.114330+0000","last_clean":"2023-10-18T17:58:23.114330+0000","last_became_active":"2023-10-18T17:58:22.701013+0000","last_became_peered":"2023-10-18T17:58:22.701013+0000","last_unstale":"2023-10-18T17:58:23.114330+0000","last_undegraded":"2023-10-18T17:58:23.114330+0000","last_fullsized":"2023-10-18T17:58:23.114330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120622+0000","last_change":"2023-10-18T17:58:22.690786+0000","last_active":"2023-10-18T17:58:23.120622+0000","last_peered":"2023-10-18T17:58:23.120622+0000","last_clean":"2023-10-18T17:58:23.120622+0000","last_became_active":"2023-10-18T17:58:22.690530+0000","last_became_peered":"2023-10-18T17:58:22.690530+0000","last_unstale":"2023-10-18T17:58:23.120622+0000","last_undegraded":"2023-10-18T17:58:23.120622+0000","last_fullsized":"2023-10-18T17:58:23.120622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121219+0000","last_change":"2023-10-18T17:58:23.112227+0000","last_active":"2023-10-18T17:58:23.121219+0000","last_peered":"2023-10-18T17:58:23.121219+0000","last_clean":"2023-10-18T17:58:23.121219+0000","last_became_active":"2023-10-18T17:58:23.111236+0000","last_became_peered":"2023-10-18T17:58:23.111236+0000","last_unstale":"2023-10-18T17:58:23.121219+0000","last_undegraded":"2023-10-18T17:58:23.121219+0000","last_fullsized":"2023-10-18T17:58:23.121219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.700834+0000","last_change":"2023-10-18T17:58:22.700039+0000","last_active":"2023-10-18T17:58:22.700834+0000","last_peered":"2023-10-18T17:58:22.700834+0000","last_clean":"2023-10-18T17:58:22.700834+0000","last_became_active":"2023-10-18T17:58:22.699719+0000","last_became_peered":"2023-10-18T17:58:22.699719+0000","last_unstale":"2023-10-18T17:58:22.700834+0000","last_undegraded":"2023-10-18T17:58:22.700834+0000","last_fullsized":"2023-10-18T17:58:22.700834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116723+0000","last_change":"2023-10-18T17:58:23.116723+0000","last_active":"2023-10-18T17:58:23.116723+0000","last_peered":"2023-10-18T17:58:23.116723+0000","last_clean":"2023-10-18T17:58:23.116723+0000","last_became_active":"2023-10-18T17:58:23.116376+0000","last_became_peered":"2023-10-18T17:58:23.116376+0000","last_unstale":"2023-10-18T17:58:23.116723+0000","last_undegraded":"2023-10-18T17:58:23.116723+0000","last_fullsized":"2023-10-18T17:58:23.116723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120037+0000","last_change":"2023-10-18T17:58:22.680018+0000","last_active":"2023-10-18T17:58:23.120037+0000","last_peered":"2023-10-18T17:58:23.120037+0000","last_clean":"2023-10-18T17:58:23.120037+0000","last_became_active":"2023-10-18T17:58:22.679464+0000","last_became_peered":"2023-10-18T17:58:22.679464+0000","last_unstale":"2023-10-18T17:58:23.120037+0000","last_undegraded":"2023-10-18T17:58:23.120037+0000","last_fullsized":"2023-10-18T17:58:23.120037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115758+0000","last_change":"2023-10-18T17:58:23.115758+0000","last_active":"2023-10-18T17:58:23.115758+0000","last_peered":"2023-10-18T17:58:23.115758+0000","last_clean":"2023-10-18T17:58:23.115758+0000","last_became_active":"2023-10-18T17:58:23.115471+0000","last_became_peered":"2023-10-18T17:58:23.115471+0000","last_unstale":"2023-10-18T17:58:23.115758+0000","last_undegraded":"2023-10-18T17:58:23.115758+0000","last_fullsized":"2023-10-18T17:58:23.115758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110821+0000","last_change":"2023-10-18T17:58:22.686370+0000","last_active":"2023-10-18T17:58:23.110821+0000","last_peered":"2023-10-18T17:58:23.110821+0000","last_clean":"2023-10-18T17:58:23.110821+0000","last_became_active":"2023-10-18T17:58:22.686062+0000","last_became_peered":"2023-10-18T17:58:22.686062+0000","last_unstale":"2023-10-18T17:58:23.110821+0000","last_undegraded":"2023-10-18T17:58:23.110821+0000","last_fullsized":"2023-10-18T17:58:23.110821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125366+0000","last_change":"2023-10-18T17:58:23.118319+0000","last_active":"2023-10-18T17:58:23.125366+0000","last_peered":"2023-10-18T17:58:23.125366+0000","last_clean":"2023-10-18T17:58:23.125366+0000","last_became_active":"2023-10-18T17:58:23.118018+0000","last_became_peered":"2023-10-18T17:58:23.118018+0000","last_unstale":"2023-10-18T17:58:23.125366+0000","last_undegraded":"2023-10-18T17:58:23.125366+0000","last_fullsized":"2023-10-18T17:58:23.125366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.698386+0000","last_change":"2023-10-18T17:58:22.697114+0000","last_active":"2023-10-18T17:58:22.698386+0000","last_peered":"2023-10-18T17:58:22.698386+0000","last_clean":"2023-10-18T17:58:22.698386+0000","last_became_active":"2023-10-18T17:58:22.696798+0000","last_became_peered":"2023-10-18T17:58:22.696798+0000","last_unstale":"2023-10-18T17:58:22.698386+0000","last_undegraded":"2023-10-18T17:58:22.698386+0000","last_fullsized":"2023-10-18T17:58:22.698386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125558+0000","last_change":"2023-10-18T17:58:23.117398+0000","last_active":"2023-10-18T17:58:23.125558+0000","last_peered":"2023-10-18T17:58:23.125558+0000","last_clean":"2023-10-18T17:58:23.125558+0000","last_became_active":"2023-10-18T17:58:23.117086+0000","last_became_peered":"2023-10-18T17:58:23.117086+0000","last_unstale":"2023-10-18T17:58:23.125558+0000","last_undegraded":"2023-10-18T17:58:23.125558+0000","last_fullsized":"2023-10-18T17:58:23.125558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103780+0000","last_change":"2023-10-18T17:58:22.678915+0000","last_active":"2023-10-18T17:58:23.103780+0000","last_peered":"2023-10-18T17:58:23.103780+0000","last_clean":"2023-10-18T17:58:23.103780+0000","last_became_active":"2023-10-18T17:58:22.678609+0000","last_became_peered":"2023-10-18T17:58:22.678609+0000","last_unstale":"2023-10-18T17:58:23.103780+0000","last_undegraded":"2023-10-18T17:58:23.103780+0000","last_fullsized":"2023-10-18T17:58:23.103780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.122008+0000","last_change":"2023-10-18T17:58:23.122008+0000","last_active":"2023-10-18T17:58:23.122008+0000","last_peered":"2023-10-18T17:58:23.122008+0000","last_clean":"2023-10-18T17:58:23.122008+0000","last_became_active":"2023-10-18T17:58:23.121684+0000","last_became_peered":"2023-10-18T17:58:23.121684+0000","last_unstale":"2023-10-18T17:58:23.122008+0000","last_undegraded":"2023-10-18T17:58:23.122008+0000","last_fullsized":"2023-10-18T17:58:23.122008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.673345+0000","last_change":"2023-10-18T17:58:22.668404+0000","last_active":"2023-10-18T17:58:22.673345+0000","last_peered":"2023-10-18T17:58:22.673345+0000","last_clean":"2023-10-18T17:58:22.673345+0000","last_became_active":"2023-10-18T17:58:22.668038+0000","last_became_peered":"2023-10-18T17:58:22.668038+0000","last_unstale":"2023-10-18T17:58:22.673345+0000","last_undegraded":"2023-10-18T17:58:22.673345+0000","last_fullsized":"2023-10-18T17:58:22.673345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127151+0000","last_change":"2023-10-18T17:58:23.121199+0000","last_active":"2023-10-18T17:58:23.127151+0000","last_peered":"2023-10-18T17:58:23.127151+0000","last_clean":"2023-10-18T17:58:23.127151+0000","last_became_active":"2023-10-18T17:58:23.120856+0000","last_became_peered":"2023-10-18T17:58:23.120856+0000","last_unstale":"2023-10-18T17:58:23.127151+0000","last_undegraded":"2023-10-18T17:58:23.127151+0000","last_fullsized":"2023-10-18T17:58:23.127151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114412+0000","last_change":"2023-10-18T17:58:22.688492+0000","last_active":"2023-10-18T17:58:23.114412+0000","last_peered":"2023-10-18T17:58:23.114412+0000","last_clean":"2023-10-18T17:58:23.114412+0000","last_became_active":"2023-10-18T17:58:22.688162+0000","last_became_peered":"2023-10-18T17:58:22.688162+0000","last_unstale":"2023-10-18T17:58:23.114412+0000","last_undegraded":"2023-10-18T17:58:23.114412+0000","last_fullsized":"2023-10-18T17:58:23.114412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125037+0000","last_change":"2023-10-18T17:58:23.125037+0000","last_active":"2023-10-18T17:58:23.125037+0000","last_peered":"2023-10-18T17:58:23.125037+0000","last_clean":"2023-10-18T17:58:23.125037+0000","last_became_active":"2023-10-18T17:58:23.124722+0000","last_became_peered":"2023-10-18T17:58:23.124722+0000","last_unstale":"2023-10-18T17:58:23.125037+0000","last_undegraded":"2023-10-18T17:58:23.125037+0000","last_fullsized":"2023-10-18T17:58:23.125037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.682504+0000","last_change":"2023-10-18T17:58:22.680359+0000","last_active":"2023-10-18T17:58:22.682504+0000","last_peered":"2023-10-18T17:58:22.682504+0000","last_clean":"2023-10-18T17:58:22.682504+0000","last_became_active":"2023-10-18T17:58:22.679356+0000","last_became_peered":"2023-10-18T17:58:22.679356+0000","last_unstale":"2023-10-18T17:58:22.682504+0000","last_undegraded":"2023-10-18T17:58:22.682504+0000","last_fullsized":"2023-10-18T17:58:22.682504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129449+0000","last_change":"2023-10-18T17:58:23.125443+0000","last_active":"2023-10-18T17:58:23.129449+0000","last_peered":"2023-10-18T17:58:23.129449+0000","last_clean":"2023-10-18T17:58:23.129449+0000","last_became_active":"2023-10-18T17:58:23.124036+0000","last_became_peered":"2023-10-18T17:58:23.124036+0000","last_unstale":"2023-10-18T17:58:23.129449+0000","last_undegraded":"2023-10-18T17:58:23.129449+0000","last_fullsized":"2023-10-18T17:58:23.129449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116673+0000","last_change":"2023-10-18T17:58:22.695695+0000","last_active":"2023-10-18T17:58:23.116673+0000","last_peered":"2023-10-18T17:58:23.116673+0000","last_clean":"2023-10-18T17:58:23.116673+0000","last_became_active":"2023-10-18T17:58:22.694956+0000","last_became_peered":"2023-10-18T17:58:22.694956+0000","last_unstale":"2023-10-18T17:58:23.116673+0000","last_undegraded":"2023-10-18T17:58:23.116673+0000","last_fullsized":"2023-10-18T17:58:23.116673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.124170+0000","last_change":"2023-10-18T17:58:23.124170+0000","last_active":"2023-10-18T17:58:23.124170+0000","last_peered":"2023-10-18T17:58:23.124170+0000","last_clean":"2023-10-18T17:58:23.124170+0000","last_became_active":"2023-10-18T17:58:23.123835+0000","last_became_peered":"2023-10-18T17:58:23.123835+0000","last_unstale":"2023-10-18T17:58:23.124170+0000","last_undegraded":"2023-10-18T17:58:23.124170+0000","last_fullsized":"2023-10-18T17:58:23.124170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683697+0000","last_change":"2023-10-18T17:58:22.678817+0000","last_active":"2023-10-18T17:58:22.683697+0000","last_peered":"2023-10-18T17:58:22.683697+0000","last_clean":"2023-10-18T17:58:22.683697+0000","last_became_active":"2023-10-18T17:58:22.678508+0000","last_became_peered":"2023-10-18T17:58:22.678508+0000","last_unstale":"2023-10-18T17:58:22.683697+0000","last_undegraded":"2023-10-18T17:58:22.683697+0000","last_fullsized":"2023-10-18T17:58:22.683697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.133265+0000","last_change":"2023-10-18T17:58:23.129445+0000","last_active":"2023-10-18T17:58:23.133265+0000","last_peered":"2023-10-18T17:58:23.133265+0000","last_clean":"2023-10-18T17:58:23.133265+0000","last_became_active":"2023-10-18T17:58:23.129038+0000","last_became_peered":"2023-10-18T17:58:23.129038+0000","last_unstale":"2023-10-18T17:58:23.133265+0000","last_undegraded":"2023-10-18T17:58:23.133265+0000","last_fullsized":"2023-10-18T17:58:23.133265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.685131+0000","last_change":"2023-10-18T17:58:22.684469+0000","last_active":"2023-10-18T17:58:22.685131+0000","last_peered":"2023-10-18T17:58:22.685131+0000","last_clean":"2023-10-18T17:58:22.685131+0000","last_became_active":"2023-10-18T17:58:22.684160+0000","last_became_peered":"2023-10-18T17:58:22.684160+0000","last_unstale":"2023-10-18T17:58:22.685131+0000","last_undegraded":"2023-10-18T17:58:22.685131+0000","last_fullsized":"2023-10-18T17:58:22.685131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120664+0000","last_change":"2023-10-18T17:58:23.120664+0000","last_active":"2023-10-18T17:58:23.120664+0000","last_peered":"2023-10-18T17:58:23.120664+0000","last_clean":"2023-10-18T17:58:23.120664+0000","last_became_active":"2023-10-18T17:58:23.120359+0000","last_became_peered":"2023-10-18T17:58:23.120359+0000","last_unstale":"2023-10-18T17:58:23.120664+0000","last_undegraded":"2023-10-18T17:58:23.120664+0000","last_fullsized":"2023-10-18T17:58:23.120664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.680709+0000","last_change":"2023-10-18T17:58:22.676645+0000","last_active":"2023-10-18T17:58:22.680709+0000","last_peered":"2023-10-18T17:58:22.680709+0000","last_clean":"2023-10-18T17:58:22.680709+0000","last_became_active":"2023-10-18T17:58:22.676295+0000","last_became_peered":"2023-10-18T17:58:22.676295+0000","last_unstale":"2023-10-18T17:58:22.680709+0000","last_undegraded":"2023-10-18T17:58:22.680709+0000","last_fullsized":"2023-10-18T17:58:22.680709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.124501+0000","last_change":"2023-10-18T17:58:23.124501+0000","last_active":"2023-10-18T17:58:23.124501+0000","last_peered":"2023-10-18T17:58:23.124501+0000","last_clean":"2023-10-18T17:58:23.124501+0000","last_became_active":"2023-10-18T17:58:23.124186+0000","last_became_peered":"2023-10-18T17:58:23.124186+0000","last_unstale":"2023-10-18T17:58:23.124501+0000","last_undegraded":"2023-10-18T17:58:23.124501+0000","last_fullsized":"2023-10-18T17:58:23.124501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117130+0000","last_change":"2023-10-18T17:58:22.690272+0000","last_active":"2023-10-18T17:58:23.117130+0000","last_peered":"2023-10-18T17:58:23.117130+0000","last_clean":"2023-10-18T17:58:23.117130+0000","last_became_active":"2023-10-18T17:58:22.689949+0000","last_became_peered":"2023-10-18T17:58:22.689949+0000","last_unstale":"2023-10-18T17:58:23.117130+0000","last_undegraded":"2023-10-18T17:58:23.117130+0000","last_fullsized":"2023-10-18T17:58:23.117130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134179+0000","last_change":"2023-10-18T17:58:23.126544+0000","last_active":"2023-10-18T17:58:23.134179+0000","last_peered":"2023-10-18T17:58:23.134179+0000","last_clean":"2023-10-18T17:58:23.134179+0000","last_became_active":"2023-10-18T17:58:23.125987+0000","last_became_peered":"2023-10-18T17:58:23.125987+0000","last_unstale":"2023-10-18T17:58:23.134179+0000","last_undegraded":"2023-10-18T17:58:23.134179+0000","last_fullsized":"2023-10-18T17:58:23.134179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.682755+0000","last_change":"2023-10-18T17:58:22.681429+0000","last_active":"2023-10-18T17:58:22.682755+0000","last_peered":"2023-10-18T17:58:22.682755+0000","last_clean":"2023-10-18T17:58:22.682755+0000","last_became_active":"2023-10-18T17:58:22.681145+0000","last_became_peered":"2023-10-18T17:58:22.681145+0000","last_unstale":"2023-10-18T17:58:22.682755+0000","last_undegraded":"2023-10-18T17:58:22.682755+0000","last_fullsized":"2023-10-18T17:58:22.682755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117504+0000","last_change":"2023-10-18T17:58:23.117504+0000","last_active":"2023-10-18T17:58:23.117504+0000","last_peered":"2023-10-18T17:58:23.117504+0000","last_clean":"2023-10-18T17:58:23.117504+0000","last_became_active":"2023-10-18T17:58:23.117192+0000","last_became_peered":"2023-10-18T17:58:23.117192+0000","last_unstale":"2023-10-18T17:58:23.117504+0000","last_undegraded":"2023-10-18T17:58:23.117504+0000","last_fullsized":"2023-10-18T17:58:23.117504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114324+0000","last_change":"2023-10-18T17:58:22.682889+0000","last_active":"2023-10-18T17:58:23.114324+0000","last_peered":"2023-10-18T17:58:23.114324+0000","last_clean":"2023-10-18T17:58:23.114324+0000","last_became_active":"2023-10-18T17:58:22.682273+0000","last_became_peered":"2023-10-18T17:58:22.682273+0000","last_unstale":"2023-10-18T17:58:23.114324+0000","last_undegraded":"2023-10-18T17:58:23.114324+0000","last_fullsized":"2023-10-18T17:58:23.114324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126856+0000","last_change":"2023-10-18T17:58:23.126856+0000","last_active":"2023-10-18T17:58:23.126856+0000","last_peered":"2023-10-18T17:58:23.126856+0000","last_clean":"2023-10-18T17:58:23.126856+0000","last_became_active":"2023-10-18T17:58:23.126582+0000","last_became_peered":"2023-10-18T17:58:23.126582+0000","last_unstale":"2023-10-18T17:58:23.126856+0000","last_undegraded":"2023-10-18T17:58:23.126856+0000","last_fullsized":"2023-10-18T17:58:23.126856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.684036+0000","last_change":"2023-10-18T17:58:22.680971+0000","last_active":"2023-10-18T17:58:22.684036+0000","last_peered":"2023-10-18T17:58:22.684036+0000","last_clean":"2023-10-18T17:58:22.684036+0000","last_became_active":"2023-10-18T17:58:22.678750+0000","last_became_peered":"2023-10-18T17:58:22.678750+0000","last_unstale":"2023-10-18T17:58:22.684036+0000","last_undegraded":"2023-10-18T17:58:22.684036+0000","last_fullsized":"2023-10-18T17:58:22.684036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135050+0000","last_change":"2023-10-18T17:58:23.130754+0000","last_active":"2023-10-18T17:58:23.135050+0000","last_peered":"2023-10-18T17:58:23.135050+0000","last_clean":"2023-10-18T17:58:23.135050+0000","last_became_active":"2023-10-18T17:58:23.130419+0000","last_became_peered":"2023-10-18T17:58:23.130419+0000","last_unstale":"2023-10-18T17:58:23.135050+0000","last_undegraded":"2023-10-18T17:58:23.135050+0000","last_fullsized":"2023-10-18T17:58:23.135050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.684724+0000","last_change":"2023-10-18T17:58:22.683291+0000","last_active":"2023-10-18T17:58:22.684724+0000","last_peered":"2023-10-18T17:58:22.684724+0000","last_clean":"2023-10-18T17:58:22.684724+0000","last_became_active":"2023-10-18T17:58:22.682926+0000","last_became_peered":"2023-10-18T17:58:22.682926+0000","last_unstale":"2023-10-18T17:58:22.684724+0000","last_undegraded":"2023-10-18T17:58:22.684724+0000","last_fullsized":"2023-10-18T17:58:22.684724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126426+0000","last_change":"2023-10-18T17:58:23.126426+0000","last_active":"2023-10-18T17:58:23.126426+0000","last_peered":"2023-10-18T17:58:23.126426+0000","last_clean":"2023-10-18T17:58:23.126426+0000","last_became_active":"2023-10-18T17:58:23.126114+0000","last_became_peered":"2023-10-18T17:58:23.126114+0000","last_unstale":"2023-10-18T17:58:23.126426+0000","last_undegraded":"2023-10-18T17:58:23.126426+0000","last_fullsized":"2023-10-18T17:58:23.126426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119826+0000","last_change":"2023-10-18T17:58:22.690514+0000","last_active":"2023-10-18T17:58:23.119826+0000","last_peered":"2023-10-18T17:58:23.119826+0000","last_clean":"2023-10-18T17:58:23.119826+0000","last_became_active":"2023-10-18T17:58:22.689427+0000","last_became_peered":"2023-10-18T17:58:22.689427+0000","last_unstale":"2023-10-18T17:58:23.119826+0000","last_undegraded":"2023-10-18T17:58:23.119826+0000","last_fullsized":"2023-10-18T17:58:23.119826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135075+0000","last_change":"2023-10-18T17:58:23.128307+0000","last_active":"2023-10-18T17:58:23.135075+0000","last_peered":"2023-10-18T17:58:23.135075+0000","last_clean":"2023-10-18T17:58:23.135075+0000","last_became_active":"2023-10-18T17:58:23.127998+0000","last_became_peered":"2023-10-18T17:58:23.127998+0000","last_unstale":"2023-10-18T17:58:23.135075+0000","last_undegraded":"2023-10-18T17:58:23.135075+0000","last_fullsized":"2023-10-18T17:58:23.135075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104658+0000","last_change":"2023-10-18T17:58:22.686163+0000","last_active":"2023-10-18T17:58:23.104658+0000","last_peered":"2023-10-18T17:58:23.104658+0000","last_clean":"2023-10-18T17:58:23.104658+0000","last_became_active":"2023-10-18T17:58:22.685950+0000","last_became_peered":"2023-10-18T17:58:22.685950+0000","last_unstale":"2023-10-18T17:58:23.104658+0000","last_undegraded":"2023-10-18T17:58:23.104658+0000","last_fullsized":"2023-10-18T17:58:23.104658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125096+0000","last_change":"2023-10-18T17:58:23.123007+0000","last_active":"2023-10-18T17:58:23.125096+0000","last_peered":"2023-10-18T17:58:23.125096+0000","last_clean":"2023-10-18T17:58:23.125096+0000","last_became_active":"2023-10-18T17:58:23.122656+0000","last_became_peered":"2023-10-18T17:58:23.122656+0000","last_unstale":"2023-10-18T17:58:23.125096+0000","last_undegraded":"2023-10-18T17:58:23.125096+0000","last_fullsized":"2023-10-18T17:58:23.125096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.691067+0000","last_change":"2023-10-18T17:58:22.690142+0000","last_active":"2023-10-18T17:58:22.691067+0000","last_peered":"2023-10-18T17:58:22.691067+0000","last_clean":"2023-10-18T17:58:22.691067+0000","last_became_active":"2023-10-18T17:58:22.689829+0000","last_became_peered":"2023-10-18T17:58:22.689829+0000","last_unstale":"2023-10-18T17:58:22.691067+0000","last_undegraded":"2023-10-18T17:58:22.691067+0000","last_fullsized":"2023-10-18T17:58:22.691067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126572+0000","last_change":"2023-10-18T17:58:23.126572+0000","last_active":"2023-10-18T17:58:23.126572+0000","last_peered":"2023-10-18T17:58:23.126572+0000","last_clean":"2023-10-18T17:58:23.126572+0000","last_became_active":"2023-10-18T17:58:23.125662+0000","last_became_peered":"2023-10-18T17:58:23.125662+0000","last_unstale":"2023-10-18T17:58:23.126572+0000","last_undegraded":"2023-10-18T17:58:23.126572+0000","last_fullsized":"2023-10-18T17:58:23.126572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103952+0000","last_change":"2023-10-18T17:58:22.674168+0000","last_active":"2023-10-18T17:58:23.103952+0000","last_peered":"2023-10-18T17:58:23.103952+0000","last_clean":"2023-10-18T17:58:23.103952+0000","last_became_active":"2023-10-18T17:58:22.668382+0000","last_became_peered":"2023-10-18T17:58:22.668382+0000","last_unstale":"2023-10-18T17:58:23.103952+0000","last_undegraded":"2023-10-18T17:58:23.103952+0000","last_fullsized":"2023-10-18T17:58:23.103952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118550+0000","last_change":"2023-10-18T17:58:23.118550+0000","last_active":"2023-10-18T17:58:23.118550+0000","last_peered":"2023-10-18T17:58:23.118550+0000","last_clean":"2023-10-18T17:58:23.118550+0000","last_became_active":"2023-10-18T17:58:23.118162+0000","last_became_peered":"2023-10-18T17:58:23.118162+0000","last_unstale":"2023-10-18T17:58:23.118550+0000","last_undegraded":"2023-10-18T17:58:23.118550+0000","last_fullsized":"2023-10-18T17:58:23.118550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681225+0000","last_change":"2023-10-18T17:58:22.678474+0000","last_active":"2023-10-18T17:58:22.681225+0000","last_peered":"2023-10-18T17:58:22.681225+0000","last_clean":"2023-10-18T17:58:22.681225+0000","last_became_active":"2023-10-18T17:58:22.678197+0000","last_became_peered":"2023-10-18T17:58:22.678197+0000","last_unstale":"2023-10-18T17:58:22.681225+0000","last_undegraded":"2023-10-18T17:58:22.681225+0000","last_fullsized":"2023-10-18T17:58:22.681225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120777+0000","last_change":"2023-10-18T17:58:23.120777+0000","last_active":"2023-10-18T17:58:23.120777+0000","last_peered":"2023-10-18T17:58:23.120777+0000","last_clean":"2023-10-18T17:58:23.120777+0000","last_became_active":"2023-10-18T17:58:23.116893+0000","last_became_peered":"2023-10-18T17:58:23.116893+0000","last_unstale":"2023-10-18T17:58:23.120777+0000","last_undegraded":"2023-10-18T17:58:23.120777+0000","last_fullsized":"2023-10-18T17:58:23.120777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681135+0000","last_change":"2023-10-18T17:58:22.677025+0000","last_active":"2023-10-18T17:58:22.681135+0000","last_peered":"2023-10-18T17:58:22.681135+0000","last_clean":"2023-10-18T17:58:22.681135+0000","last_became_active":"2023-10-18T17:58:22.676536+0000","last_became_peered":"2023-10-18T17:58:22.676536+0000","last_unstale":"2023-10-18T17:58:22.681135+0000","last_undegraded":"2023-10-18T17:58:22.681135+0000","last_fullsized":"2023-10-18T17:58:22.681135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129785+0000","last_change":"2023-10-18T17:58:23.121230+0000","last_active":"2023-10-18T17:58:23.129785+0000","last_peered":"2023-10-18T17:58:23.129785+0000","last_clean":"2023-10-18T17:58:23.129785+0000","last_became_active":"2023-10-18T17:58:23.120977+0000","last_became_peered":"2023-10-18T17:58:23.120977+0000","last_unstale":"2023-10-18T17:58:23.129785+0000","last_undegraded":"2023-10-18T17:58:23.129785+0000","last_fullsized":"2023-10-18T17:58:23.129785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.692413+0000","last_change":"2023-10-18T17:58:22.691513+0000","last_active":"2023-10-18T17:58:22.692413+0000","last_peered":"2023-10-18T17:58:22.692413+0000","last_clean":"2023-10-18T17:58:22.692413+0000","last_became_active":"2023-10-18T17:58:22.691220+0000","last_became_peered":"2023-10-18T17:58:22.691220+0000","last_unstale":"2023-10-18T17:58:22.692413+0000","last_undegraded":"2023-10-18T17:58:22.692413+0000","last_fullsized":"2023-10-18T17:58:22.692413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128454+0000","last_change":"2023-10-18T17:58:23.128454+0000","last_active":"2023-10-18T17:58:23.128454+0000","last_peered":"2023-10-18T17:58:23.128454+0000","last_clean":"2023-10-18T17:58:23.128454+0000","last_became_active":"2023-10-18T17:58:23.128113+0000","last_became_peered":"2023-10-18T17:58:23.128113+0000","last_unstale":"2023-10-18T17:58:23.128454+0000","last_undegraded":"2023-10-18T17:58:23.128454+0000","last_fullsized":"2023-10-18T17:58:23.128454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116763+0000","last_change":"2023-10-18T17:58:22.695883+0000","last_active":"2023-10-18T17:58:23.116763+0000","last_peered":"2023-10-18T17:58:23.116763+0000","last_clean":"2023-10-18T17:58:23.116763+0000","last_became_active":"2023-10-18T17:58:22.695183+0000","last_became_peered":"2023-10-18T17:58:22.695183+0000","last_unstale":"2023-10-18T17:58:23.116763+0000","last_undegraded":"2023-10-18T17:58:23.116763+0000","last_fullsized":"2023-10-18T17:58:23.116763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.123828+0000","last_change":"2023-10-18T17:58:23.123828+0000","last_active":"2023-10-18T17:58:23.123828+0000","last_peered":"2023-10-18T17:58:23.123828+0000","last_clean":"2023-10-18T17:58:23.123828+0000","last_became_active":"2023-10-18T17:58:23.123552+0000","last_became_peered":"2023-10-18T17:58:23.123552+0000","last_unstale":"2023-10-18T17:58:23.123828+0000","last_undegraded":"2023-10-18T17:58:23.123828+0000","last_fullsized":"2023-10-18T17:58:23.123828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111182+0000","last_change":"2023-10-18T17:58:22.685450+0000","last_active":"2023-10-18T17:58:23.111182+0000","last_peered":"2023-10-18T17:58:23.111182+0000","last_clean":"2023-10-18T17:58:23.111182+0000","last_became_active":"2023-10-18T17:58:22.685168+0000","last_became_peered":"2023-10-18T17:58:22.685168+0000","last_unstale":"2023-10-18T17:58:23.111182+0000","last_undegraded":"2023-10-18T17:58:23.111182+0000","last_fullsized":"2023-10-18T17:58:23.111182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114901+0000","last_change":"2023-10-18T17:58:23.114901+0000","last_active":"2023-10-18T17:58:23.114901+0000","last_peered":"2023-10-18T17:58:23.114901+0000","last_clean":"2023-10-18T17:58:23.114901+0000","last_became_active":"2023-10-18T17:58:23.114590+0000","last_became_peered":"2023-10-18T17:58:23.114590+0000","last_unstale":"2023-10-18T17:58:23.114901+0000","last_undegraded":"2023-10-18T17:58:23.114901+0000","last_fullsized":"2023-10-18T17:58:23.114901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112133+0000","last_change":"2023-10-18T17:58:22.692159+0000","last_active":"2023-10-18T17:58:23.112133+0000","last_peered":"2023-10-18T17:58:23.112133+0000","last_clean":"2023-10-18T17:58:23.112133+0000","last_became_active":"2023-10-18T17:58:22.691846+0000","last_became_peered":"2023-10-18T17:58:22.691846+0000","last_unstale":"2023-10-18T17:58:23.112133+0000","last_undegraded":"2023-10-18T17:58:23.112133+0000","last_fullsized":"2023-10-18T17:58:23.112133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136653+0000","last_change":"2023-10-18T17:58:23.134892+0000","last_active":"2023-10-18T17:58:23.136653+0000","last_peered":"2023-10-18T17:58:23.136653+0000","last_clean":"2023-10-18T17:58:23.136653+0000","last_became_active":"2023-10-18T17:58:23.134213+0000","last_became_peered":"2023-10-18T17:58:23.134213+0000","last_unstale":"2023-10-18T17:58:23.136653+0000","last_undegraded":"2023-10-18T17:58:23.136653+0000","last_fullsized":"2023-10-18T17:58:23.136653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120693+0000","last_change":"2023-10-18T17:58:17.791475+0000","last_active":"2023-10-18T17:58:23.120693+0000","last_peered":"2023-10-18T17:58:23.120693+0000","last_clean":"2023-10-18T17:58:23.120693+0000","last_became_active":"2023-10-18T17:58:17.790774+0000","last_became_peered":"2023-10-18T17:58:17.790774+0000","last_unstale":"2023-10-18T17:58:23.120693+0000","last_undegraded":"2023-10-18T17:58:23.120693+0000","last_fullsized":"2023-10-18T17:58:23.120693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681177+0000","last_change":"2023-10-18T17:58:22.676456+0000","last_active":"2023-10-18T17:58:22.681177+0000","last_peered":"2023-10-18T17:58:22.681177+0000","last_clean":"2023-10-18T17:58:22.681177+0000","last_became_active":"2023-10-18T17:58:22.676189+0000","last_became_peered":"2023-10-18T17:58:22.676189+0000","last_unstale":"2023-10-18T17:58:22.681177+0000","last_undegraded":"2023-10-18T17:58:22.681177+0000","last_fullsized":"2023-10-18T17:58:22.681177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136521+0000","last_change":"2023-10-18T17:58:23.133571+0000","last_active":"2023-10-18T17:58:23.136521+0000","last_peered":"2023-10-18T17:58:23.136521+0000","last_clean":"2023-10-18T17:58:23.136521+0000","last_became_active":"2023-10-18T17:58:23.133302+0000","last_became_peered":"2023-10-18T17:58:23.133302+0000","last_unstale":"2023-10-18T17:58:23.136521+0000","last_undegraded":"2023-10-18T17:58:23.136521+0000","last_fullsized":"2023-10-18T17:58:23.136521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.665338+0000","last_change":"2023-10-18T17:58:17.792368+0000","last_active":"2023-10-18T17:58:22.665338+0000","last_peered":"2023-10-18T17:58:22.665338+0000","last_clean":"2023-10-18T17:58:22.665338+0000","last_became_active":"2023-10-18T17:58:17.791751+0000","last_became_peered":"2023-10-18T17:58:17.791751+0000","last_unstale":"2023-10-18T17:58:22.665338+0000","last_undegraded":"2023-10-18T17:58:22.665338+0000","last_fullsized":"2023-10-18T17:58:22.665338+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.679239+0000","last_change":"2023-10-18T17:58:22.675348+0000","last_active":"2023-10-18T17:58:22.679239+0000","last_peered":"2023-10-18T17:58:22.679239+0000","last_clean":"2023-10-18T17:58:22.679239+0000","last_became_active":"2023-10-18T17:58:22.674469+0000","last_became_peered":"2023-10-18T17:58:22.674469+0000","last_unstale":"2023-10-18T17:58:22.679239+0000","last_undegraded":"2023-10-18T17:58:22.679239+0000","last_fullsized":"2023-10-18T17:58:22.679239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121217+0000","last_change":"2023-10-18T17:58:23.121217+0000","last_active":"2023-10-18T17:58:23.121217+0000","last_peered":"2023-10-18T17:58:23.121217+0000","last_clean":"2023-10-18T17:58:23.121217+0000","last_became_active":"2023-10-18T17:58:23.120691+0000","last_became_peered":"2023-10-18T17:58:23.120691+0000","last_unstale":"2023-10-18T17:58:23.121217+0000","last_undegraded":"2023-10-18T17:58:23.121217+0000","last_fullsized":"2023-10-18T17:58:23.121217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120107+0000","last_change":"2023-10-18T17:58:17.605810+0000","last_active":"2023-10-18T17:58:23.120107+0000","last_peered":"2023-10-18T17:58:23.120107+0000","last_clean":"2023-10-18T17:58:23.120107+0000","last_became_active":"2023-10-18T17:58:17.605289+0000","last_became_peered":"2023-10-18T17:58:17.605289+0000","last_unstale":"2023-10-18T17:58:23.120107+0000","last_undegraded":"2023-10-18T17:58:23.120107+0000","last_fullsized":"2023-10-18T17:58:23.120107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681523+0000","last_change":"2023-10-18T17:58:22.675274+0000","last_active":"2023-10-18T17:58:22.681523+0000","last_peered":"2023-10-18T17:58:22.681523+0000","last_clean":"2023-10-18T17:58:22.681523+0000","last_became_active":"2023-10-18T17:58:22.674965+0000","last_became_peered":"2023-10-18T17:58:22.674965+0000","last_unstale":"2023-10-18T17:58:22.681523+0000","last_undegraded":"2023-10-18T17:58:22.681523+0000","last_fullsized":"2023-10-18T17:58:22.681523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125547+0000","last_change":"2023-10-18T17:58:23.125547+0000","last_active":"2023-10-18T17:58:23.125547+0000","last_peered":"2023-10-18T17:58:23.125547+0000","last_clean":"2023-10-18T17:58:23.125547+0000","last_became_active":"2023-10-18T17:58:23.125259+0000","last_became_peered":"2023-10-18T17:58:23.125259+0000","last_unstale":"2023-10-18T17:58:23.125547+0000","last_undegraded":"2023-10-18T17:58:23.125547+0000","last_fullsized":"2023-10-18T17:58:23.125547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.662363+0000","last_change":"2023-10-18T17:58:18.326048+0000","last_active":"2023-10-18T17:58:22.662363+0000","last_peered":"2023-10-18T17:58:22.662363+0000","last_clean":"2023-10-18T17:58:22.662363+0000","last_became_active":"2023-10-18T17:58:18.325141+0000","last_became_peered":"2023-10-18T17:58:18.325141+0000","last_unstale":"2023-10-18T17:58:22.662363+0000","last_undegraded":"2023-10-18T17:58:22.662363+0000","last_fullsized":"2023-10-18T17:58:22.662363+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114331+0000","last_change":"2023-10-18T17:58:22.684816+0000","last_active":"2023-10-18T17:58:23.114331+0000","last_peered":"2023-10-18T17:58:23.114331+0000","last_clean":"2023-10-18T17:58:23.114331+0000","last_became_active":"2023-10-18T17:58:22.682458+0000","last_became_peered":"2023-10-18T17:58:22.682458+0000","last_unstale":"2023-10-18T17:58:23.114331+0000","last_undegraded":"2023-10-18T17:58:23.114331+0000","last_fullsized":"2023-10-18T17:58:23.114331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.131050+0000","last_change":"2023-10-18T17:58:23.131050+0000","last_active":"2023-10-18T17:58:23.131050+0000","last_peered":"2023-10-18T17:58:23.131050+0000","last_clean":"2023-10-18T17:58:23.131050+0000","last_became_active":"2023-10-18T17:58:23.130768+0000","last_became_peered":"2023-10-18T17:58:23.130768+0000","last_unstale":"2023-10-18T17:58:23.131050+0000","last_undegraded":"2023-10-18T17:58:23.131050+0000","last_fullsized":"2023-10-18T17:58:23.131050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110585+0000","last_change":"2023-10-18T17:58:22.698814+0000","last_active":"2023-10-18T17:58:23.110585+0000","last_peered":"2023-10-18T17:58:23.110585+0000","last_clean":"2023-10-18T17:58:23.110585+0000","last_became_active":"2023-10-18T17:58:22.698525+0000","last_became_peered":"2023-10-18T17:58:22.698525+0000","last_unstale":"2023-10-18T17:58:23.110585+0000","last_undegraded":"2023-10-18T17:58:23.110585+0000","last_fullsized":"2023-10-18T17:58:23.110585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117324+0000","last_change":"2023-10-18T17:58:18.322815+0000","last_active":"2023-10-18T17:58:23.117324+0000","last_peered":"2023-10-18T17:58:23.117324+0000","last_clean":"2023-10-18T17:58:23.117324+0000","last_became_active":"2023-10-18T17:58:18.322057+0000","last_became_peered":"2023-10-18T17:58:18.322057+0000","last_unstale":"2023-10-18T17:58:23.117324+0000","last_undegraded":"2023-10-18T17:58:23.117324+0000","last_fullsized":"2023-10-18T17:58:23.117324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125081+0000","last_change":"2023-10-18T17:58:23.125081+0000","last_active":"2023-10-18T17:58:23.125081+0000","last_peered":"2023-10-18T17:58:23.125081+0000","last_clean":"2023-10-18T17:58:23.125081+0000","last_became_active":"2023-10-18T17:58:23.124833+0000","last_became_peered":"2023-10-18T17:58:23.124833+0000","last_unstale":"2023-10-18T17:58:23.125081+0000","last_undegraded":"2023-10-18T17:58:23.125081+0000","last_fullsized":"2023-10-18T17:58:23.125081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683859+0000","last_change":"2023-10-18T17:58:22.680737+0000","last_active":"2023-10-18T17:58:22.683859+0000","last_peered":"2023-10-18T17:58:22.683859+0000","last_clean":"2023-10-18T17:58:22.683859+0000","last_became_active":"2023-10-18T17:58:22.676981+0000","last_became_peered":"2023-10-18T17:58:22.676981+0000","last_unstale":"2023-10-18T17:58:22.683859+0000","last_undegraded":"2023-10-18T17:58:22.683859+0000","last_fullsized":"2023-10-18T17:58:22.683859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.669846+0000","last_change":"2023-10-18T17:58:17.608107+0000","last_active":"2023-10-18T17:58:22.669846+0000","last_peered":"2023-10-18T17:58:22.669846+0000","last_clean":"2023-10-18T17:58:22.669846+0000","last_became_active":"2023-10-18T17:58:17.607603+0000","last_became_peered":"2023-10-18T17:58:17.607603+0000","last_unstale":"2023-10-18T17:58:22.669846+0000","last_undegraded":"2023-10-18T17:58:22.669846+0000","last_fullsized":"2023-10-18T17:58:22.669846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125950+0000","last_change":"2023-10-18T17:58:23.125950+0000","last_active":"2023-10-18T17:58:23.125950+0000","last_peered":"2023-10-18T17:58:23.125950+0000","last_clean":"2023-10-18T17:58:23.125950+0000","last_became_active":"2023-10-18T17:58:23.125172+0000","last_became_peered":"2023-10-18T17:58:23.125172+0000","last_unstale":"2023-10-18T17:58:23.125950+0000","last_undegraded":"2023-10-18T17:58:23.125950+0000","last_fullsized":"2023-10-18T17:58:23.125950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.684584+0000","last_change":"2023-10-18T17:58:22.683261+0000","last_active":"2023-10-18T17:58:22.684584+0000","last_peered":"2023-10-18T17:58:22.684584+0000","last_clean":"2023-10-18T17:58:22.684584+0000","last_became_active":"2023-10-18T17:58:22.682927+0000","last_became_peered":"2023-10-18T17:58:22.682927+0000","last_unstale":"2023-10-18T17:58:22.684584+0000","last_undegraded":"2023-10-18T17:58:22.684584+0000","last_fullsized":"2023-10-18T17:58:22.684584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114484+0000","last_change":"2023-10-18T17:58:17.607313+0000","last_active":"2023-10-18T17:58:23.114484+0000","last_peered":"2023-10-18T17:58:23.114484+0000","last_clean":"2023-10-18T17:58:23.114484+0000","last_became_active":"2023-10-18T17:58:17.606933+0000","last_became_peered":"2023-10-18T17:58:17.606933+0000","last_unstale":"2023-10-18T17:58:23.114484+0000","last_undegraded":"2023-10-18T17:58:23.114484+0000","last_fullsized":"2023-10-18T17:58:23.114484+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116455+0000","last_change":"2023-10-18T17:58:23.116455+0000","last_active":"2023-10-18T17:58:23.116455+0000","last_peered":"2023-10-18T17:58:23.116455+0000","last_clean":"2023-10-18T17:58:23.116455+0000","last_became_active":"2023-10-18T17:58:23.116050+0000","last_became_peered":"2023-10-18T17:58:23.116050+0000","last_unstale":"2023-10-18T17:58:23.116455+0000","last_undegraded":"2023-10-18T17:58:23.116455+0000","last_fullsized":"2023-10-18T17:58:23.116455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.690614+0000","last_change":"2023-10-18T17:58:22.689750+0000","last_active":"2023-10-18T17:58:22.690614+0000","last_peered":"2023-10-18T17:58:22.690614+0000","last_clean":"2023-10-18T17:58:22.690614+0000","last_became_active":"2023-10-18T17:58:22.689474+0000","last_became_peered":"2023-10-18T17:58:22.689474+0000","last_unstale":"2023-10-18T17:58:22.690614+0000","last_undegraded":"2023-10-18T17:58:22.690614+0000","last_fullsized":"2023-10-18T17:58:22.690614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112122+0000","last_change":"2023-10-18T17:58:17.646977+0000","last_active":"2023-10-18T17:58:23.112122+0000","last_peered":"2023-10-18T17:58:23.112122+0000","last_clean":"2023-10-18T17:58:23.112122+0000","last_became_active":"2023-10-18T17:58:17.646568+0000","last_became_peered":"2023-10-18T17:58:17.646568+0000","last_unstale":"2023-10-18T17:58:23.112122+0000","last_undegraded":"2023-10-18T17:58:23.112122+0000","last_fullsized":"2023-10-18T17:58:23.112122+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.101935+0000","last_change":"2023-10-18T17:58:15.581426+0000","last_active":"2023-10-18T17:58:23.101935+0000","last_peered":"2023-10-18T17:58:23.101935+0000","last_clean":"2023-10-18T17:58:23.101935+0000","last_became_active":"2023-10-18T17:58:15.580900+0000","last_became_peered":"2023-10-18T17:58:15.580900+0000","last_unstale":"2023-10-18T17:58:23.101935+0000","last_undegraded":"2023-10-18T17:58:23.101935+0000","last_fullsized":"2023-10-18T17:58:23.101935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:14.550597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:14.550597+0000","last_clean_scrub_stamp":"2023-10-18T17:58:14.550597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135976+0000","last_change":"2023-10-18T17:58:23.130365+0000","last_active":"2023-10-18T17:58:23.135976+0000","last_peered":"2023-10-18T17:58:23.135976+0000","last_clean":"2023-10-18T17:58:23.135976+0000","last_became_active":"2023-10-18T17:58:23.128150+0000","last_became_peered":"2023-10-18T17:58:23.128150+0000","last_unstale":"2023-10-18T17:58:23.135976+0000","last_undegraded":"2023-10-18T17:58:23.135976+0000","last_fullsized":"2023-10-18T17:58:23.135976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681134+0000","last_change":"2023-10-18T17:58:22.675737+0000","last_active":"2023-10-18T17:58:22.681134+0000","last_peered":"2023-10-18T17:58:22.681134+0000","last_clean":"2023-10-18T17:58:22.681134+0000","last_became_active":"2023-10-18T17:58:22.675306+0000","last_became_peered":"2023-10-18T17:58:22.675306+0000","last_unstale":"2023-10-18T17:58:22.681134+0000","last_undegraded":"2023-10-18T17:58:22.681134+0000","last_fullsized":"2023-10-18T17:58:22.681134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115745+0000","last_change":"2023-10-18T17:58:23.115745+0000","last_active":"2023-10-18T17:58:23.115745+0000","last_peered":"2023-10-18T17:58:23.115745+0000","last_clean":"2023-10-18T17:58:23.115745+0000","last_became_active":"2023-10-18T17:58:23.115454+0000","last_became_peered":"2023-10-18T17:58:23.115454+0000","last_unstale":"2023-10-18T17:58:23.115745+0000","last_undegraded":"2023-10-18T17:58:23.115745+0000","last_fullsized":"2023-10-18T17:58:23.115745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.113848+0000","last_change":"2023-10-18T17:58:22.690810+0000","last_active":"2023-10-18T17:58:23.113848+0000","last_peered":"2023-10-18T17:58:23.113848+0000","last_clean":"2023-10-18T17:58:23.113848+0000","last_became_active":"2023-10-18T17:58:22.690506+0000","last_became_peered":"2023-10-18T17:58:22.690506+0000","last_unstale":"2023-10-18T17:58:23.113848+0000","last_undegraded":"2023-10-18T17:58:23.113848+0000","last_fullsized":"2023-10-18T17:58:23.113848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.133051+0000","last_change":"2023-10-18T17:58:23.126643+0000","last_active":"2023-10-18T17:58:23.133051+0000","last_peered":"2023-10-18T17:58:23.133051+0000","last_clean":"2023-10-18T17:58:23.133051+0000","last_became_active":"2023-10-18T17:58:23.126371+0000","last_became_peered":"2023-10-18T17:58:23.126371+0000","last_unstale":"2023-10-18T17:58:23.133051+0000","last_undegraded":"2023-10-18T17:58:23.133051+0000","last_fullsized":"2023-10-18T17:58:23.133051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117250+0000","last_change":"2023-10-18T17:58:22.690893+0000","last_active":"2023-10-18T17:58:23.117250+0000","last_peered":"2023-10-18T17:58:23.117250+0000","last_clean":"2023-10-18T17:58:23.117250+0000","last_became_active":"2023-10-18T17:58:22.689078+0000","last_became_peered":"2023-10-18T17:58:22.689078+0000","last_unstale":"2023-10-18T17:58:23.117250+0000","last_undegraded":"2023-10-18T17:58:23.117250+0000","last_fullsized":"2023-10-18T17:58:23.117250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135993+0000","last_change":"2023-10-18T17:58:23.130578+0000","last_active":"2023-10-18T17:58:23.135993+0000","last_peered":"2023-10-18T17:58:23.135993+0000","last_clean":"2023-10-18T17:58:23.135993+0000","last_became_active":"2023-10-18T17:58:23.130103+0000","last_became_peered":"2023-10-18T17:58:23.130103+0000","last_unstale":"2023-10-18T17:58:23.135993+0000","last_undegraded":"2023-10-18T17:58:23.135993+0000","last_fullsized":"2023-10-18T17:58:23.135993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683951+0000","last_change":"2023-10-18T17:58:22.682353+0000","last_active":"2023-10-18T17:58:22.683951+0000","last_peered":"2023-10-18T17:58:22.683951+0000","last_clean":"2023-10-18T17:58:22.683951+0000","last_became_active":"2023-10-18T17:58:22.682040+0000","last_became_peered":"2023-10-18T17:58:22.682040+0000","last_unstale":"2023-10-18T17:58:22.683951+0000","last_undegraded":"2023-10-18T17:58:22.683951+0000","last_fullsized":"2023-10-18T17:58:22.683951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127708+0000","last_change":"2023-10-18T17:58:23.120218+0000","last_active":"2023-10-18T17:58:23.127708+0000","last_peered":"2023-10-18T17:58:23.127708+0000","last_clean":"2023-10-18T17:58:23.127708+0000","last_became_active":"2023-10-18T17:58:23.119968+0000","last_became_peered":"2023-10-18T17:58:23.119968+0000","last_unstale":"2023-10-18T17:58:23.127708+0000","last_undegraded":"2023-10-18T17:58:23.127708+0000","last_fullsized":"2023-10-18T17:58:23.127708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110949+0000","last_change":"2023-10-18T17:58:22.679610+0000","last_active":"2023-10-18T17:58:23.110949+0000","last_peered":"2023-10-18T17:58:23.110949+0000","last_clean":"2023-10-18T17:58:23.110949+0000","last_became_active":"2023-10-18T17:58:22.678880+0000","last_became_peered":"2023-10-18T17:58:22.678880+0000","last_unstale":"2023-10-18T17:58:23.110949+0000","last_undegraded":"2023-10-18T17:58:23.110949+0000","last_fullsized":"2023-10-18T17:58:23.110949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118035+0000","last_change":"2023-10-18T17:58:23.118035+0000","last_active":"2023-10-18T17:58:23.118035+0000","last_peered":"2023-10-18T17:58:23.118035+0000","last_clean":"2023-10-18T17:58:23.118035+0000","last_became_active":"2023-10-18T17:58:23.117448+0000","last_became_peered":"2023-10-18T17:58:23.117448+0000","last_unstale":"2023-10-18T17:58:23.118035+0000","last_undegraded":"2023-10-18T17:58:23.118035+0000","last_fullsized":"2023-10-18T17:58:23.118035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119936+0000","last_change":"2023-10-18T17:58:22.688810+0000","last_active":"2023-10-18T17:58:23.119936+0000","last_peered":"2023-10-18T17:58:23.119936+0000","last_clean":"2023-10-18T17:58:23.119936+0000","last_became_active":"2023-10-18T17:58:22.688224+0000","last_became_peered":"2023-10-18T17:58:22.688224+0000","last_unstale":"2023-10-18T17:58:23.119936+0000","last_undegraded":"2023-10-18T17:58:23.119936+0000","last_fullsized":"2023-10-18T17:58:23.119936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136556+0000","last_change":"2023-10-18T17:58:23.133896+0000","last_active":"2023-10-18T17:58:23.136556+0000","last_peered":"2023-10-18T17:58:23.136556+0000","last_clean":"2023-10-18T17:58:23.136556+0000","last_became_active":"2023-10-18T17:58:23.133588+0000","last_became_peered":"2023-10-18T17:58:23.133588+0000","last_unstale":"2023-10-18T17:58:23.136556+0000","last_undegraded":"2023-10-18T17:58:23.136556+0000","last_fullsized":"2023-10-18T17:58:23.136556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116851+0000","last_change":"2023-10-18T17:58:22.696491+0000","last_active":"2023-10-18T17:58:23.116851+0000","last_peered":"2023-10-18T17:58:23.116851+0000","last_clean":"2023-10-18T17:58:23.116851+0000","last_became_active":"2023-10-18T17:58:22.696171+0000","last_became_peered":"2023-10-18T17:58:22.696171+0000","last_unstale":"2023-10-18T17:58:23.116851+0000","last_undegraded":"2023-10-18T17:58:23.116851+0000","last_fullsized":"2023-10-18T17:58:23.116851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125405+0000","last_change":"2023-10-18T17:58:23.116692+0000","last_active":"2023-10-18T17:58:23.125405+0000","last_peered":"2023-10-18T17:58:23.125405+0000","last_clean":"2023-10-18T17:58:23.125405+0000","last_became_active":"2023-10-18T17:58:23.116431+0000","last_became_peered":"2023-10-18T17:58:23.116431+0000","last_unstale":"2023-10-18T17:58:23.125405+0000","last_undegraded":"2023-10-18T17:58:23.125405+0000","last_fullsized":"2023-10-18T17:58:23.125405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111957+0000","last_change":"2023-10-18T17:58:22.694825+0000","last_active":"2023-10-18T17:58:23.111957+0000","last_peered":"2023-10-18T17:58:23.111957+0000","last_clean":"2023-10-18T17:58:23.111957+0000","last_became_active":"2023-10-18T17:58:22.694563+0000","last_became_peered":"2023-10-18T17:58:22.694563+0000","last_unstale":"2023-10-18T17:58:23.111957+0000","last_undegraded":"2023-10-18T17:58:23.111957+0000","last_fullsized":"2023-10-18T17:58:23.111957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126937+0000","last_change":"2023-10-18T17:58:23.120769+0000","last_active":"2023-10-18T17:58:23.126937+0000","last_peered":"2023-10-18T17:58:23.126937+0000","last_clean":"2023-10-18T17:58:23.126937+0000","last_became_active":"2023-10-18T17:58:23.120464+0000","last_became_peered":"2023-10-18T17:58:23.120464+0000","last_unstale":"2023-10-18T17:58:23.126937+0000","last_undegraded":"2023-10-18T17:58:23.126937+0000","last_fullsized":"2023-10-18T17:58:23.126937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.101806+0000","last_change":"2023-10-18T17:58:22.668326+0000","last_active":"2023-10-18T17:58:23.101806+0000","last_peered":"2023-10-18T17:58:23.101806+0000","last_clean":"2023-10-18T17:58:23.101806+0000","last_became_active":"2023-10-18T17:58:22.668059+0000","last_became_peered":"2023-10-18T17:58:22.668059+0000","last_unstale":"2023-10-18T17:58:23.101806+0000","last_undegraded":"2023-10-18T17:58:23.101806+0000","last_fullsized":"2023-10-18T17:58:23.101806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120596+0000","last_change":"2023-10-18T17:58:23.120596+0000","last_active":"2023-10-18T17:58:23.120596+0000","last_peered":"2023-10-18T17:58:23.120596+0000","last_clean":"2023-10-18T17:58:23.120596+0000","last_became_active":"2023-10-18T17:58:23.120309+0000","last_became_peered":"2023-10-18T17:58:23.120309+0000","last_unstale":"2023-10-18T17:58:23.120596+0000","last_undegraded":"2023-10-18T17:58:23.120596+0000","last_fullsized":"2023-10-18T17:58:23.120596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.113755+0000","last_change":"2023-10-18T17:58:22.700021+0000","last_active":"2023-10-18T17:58:23.113755+0000","last_peered":"2023-10-18T17:58:23.113755+0000","last_clean":"2023-10-18T17:58:23.113755+0000","last_became_active":"2023-10-18T17:58:22.699570+0000","last_became_peered":"2023-10-18T17:58:22.699570+0000","last_unstale":"2023-10-18T17:58:23.113755+0000","last_undegraded":"2023-10-18T17:58:23.113755+0000","last_fullsized":"2023-10-18T17:58:23.113755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118771+0000","last_change":"2023-10-18T17:58:23.118771+0000","last_active":"2023-10-18T17:58:23.118771+0000","last_peered":"2023-10-18T17:58:23.118771+0000","last_clean":"2023-10-18T17:58:23.118771+0000","last_became_active":"2023-10-18T17:58:23.118493+0000","last_became_peered":"2023-10-18T17:58:23.118493+0000","last_unstale":"2023-10-18T17:58:23.118771+0000","last_undegraded":"2023-10-18T17:58:23.118771+0000","last_fullsized":"2023-10-18T17:58:23.118771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.695181+0000","last_change":"2023-10-18T17:58:22.694576+0000","last_active":"2023-10-18T17:58:22.695181+0000","last_peered":"2023-10-18T17:58:22.695181+0000","last_clean":"2023-10-18T17:58:22.695181+0000","last_became_active":"2023-10-18T17:58:22.694240+0000","last_became_peered":"2023-10-18T17:58:22.694240+0000","last_unstale":"2023-10-18T17:58:22.695181+0000","last_undegraded":"2023-10-18T17:58:22.695181+0000","last_fullsized":"2023-10-18T17:58:22.695181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134198+0000","last_change":"2023-10-18T17:58:23.126726+0000","last_active":"2023-10-18T17:58:23.134198+0000","last_peered":"2023-10-18T17:58:23.134198+0000","last_clean":"2023-10-18T17:58:23.134198+0000","last_became_active":"2023-10-18T17:58:23.126297+0000","last_became_peered":"2023-10-18T17:58:23.126297+0000","last_unstale":"2023-10-18T17:58:23.134198+0000","last_undegraded":"2023-10-18T17:58:23.134198+0000","last_fullsized":"2023-10-18T17:58:23.134198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112044+0000","last_change":"2023-10-18T17:58:22.690048+0000","last_active":"2023-10-18T17:58:23.112044+0000","last_peered":"2023-10-18T17:58:23.112044+0000","last_clean":"2023-10-18T17:58:23.112044+0000","last_became_active":"2023-10-18T17:58:22.689785+0000","last_became_peered":"2023-10-18T17:58:22.689785+0000","last_unstale":"2023-10-18T17:58:23.112044+0000","last_undegraded":"2023-10-18T17:58:23.112044+0000","last_fullsized":"2023-10-18T17:58:23.112044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129919+0000","last_change":"2023-10-18T17:58:23.129919+0000","last_active":"2023-10-18T17:58:23.129919+0000","last_peered":"2023-10-18T17:58:23.129919+0000","last_clean":"2023-10-18T17:58:23.129919+0000","last_became_active":"2023-10-18T17:58:23.129560+0000","last_became_peered":"2023-10-18T17:58:23.129560+0000","last_unstale":"2023-10-18T17:58:23.129919+0000","last_undegraded":"2023-10-18T17:58:23.129919+0000","last_fullsized":"2023-10-18T17:58:23.129919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683706+0000","last_change":"2023-10-18T17:58:22.678801+0000","last_active":"2023-10-18T17:58:22.683706+0000","last_peered":"2023-10-18T17:58:22.683706+0000","last_clean":"2023-10-18T17:58:22.683706+0000","last_became_active":"2023-10-18T17:58:22.678493+0000","last_became_peered":"2023-10-18T17:58:22.678493+0000","last_unstale":"2023-10-18T17:58:22.683706+0000","last_undegraded":"2023-10-18T17:58:22.683706+0000","last_fullsized":"2023-10-18T17:58:22.683706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129374+0000","last_change":"2023-10-18T17:58:23.124984+0000","last_active":"2023-10-18T17:58:23.129374+0000","last_peered":"2023-10-18T17:58:23.129374+0000","last_clean":"2023-10-18T17:58:23.129374+0000","last_became_active":"2023-10-18T17:58:23.124646+0000","last_became_peered":"2023-10-18T17:58:23.124646+0000","last_unstale":"2023-10-18T17:58:23.129374+0000","last_undegraded":"2023-10-18T17:58:23.129374+0000","last_fullsized":"2023-10-18T17:58:23.129374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.681640+0000","last_change":"2023-10-18T17:58:22.677885+0000","last_active":"2023-10-18T17:58:22.681640+0000","last_peered":"2023-10-18T17:58:22.681640+0000","last_clean":"2023-10-18T17:58:22.681640+0000","last_became_active":"2023-10-18T17:58:22.677592+0000","last_became_peered":"2023-10-18T17:58:22.677592+0000","last_unstale":"2023-10-18T17:58:22.681640+0000","last_undegraded":"2023-10-18T17:58:22.681640+0000","last_fullsized":"2023-10-18T17:58:22.681640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115290+0000","last_change":"2023-10-18T17:58:23.115290+0000","last_active":"2023-10-18T17:58:23.115290+0000","last_peered":"2023-10-18T17:58:23.115290+0000","last_clean":"2023-10-18T17:58:23.115290+0000","last_became_active":"2023-10-18T17:58:23.115014+0000","last_became_peered":"2023-10-18T17:58:23.115014+0000","last_unstale":"2023-10-18T17:58:23.115290+0000","last_undegraded":"2023-10-18T17:58:23.115290+0000","last_fullsized":"2023-10-18T17:58:23.115290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110719+0000","last_change":"2023-10-18T17:58:22.695980+0000","last_active":"2023-10-18T17:58:23.110719+0000","last_peered":"2023-10-18T17:58:23.110719+0000","last_clean":"2023-10-18T17:58:23.110719+0000","last_became_active":"2023-10-18T17:58:22.695592+0000","last_became_peered":"2023-10-18T17:58:22.695592+0000","last_unstale":"2023-10-18T17:58:23.110719+0000","last_undegraded":"2023-10-18T17:58:23.110719+0000","last_fullsized":"2023-10-18T17:58:23.110719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.124321+0000","last_change":"2023-10-18T17:58:23.124321+0000","last_active":"2023-10-18T17:58:23.124321+0000","last_peered":"2023-10-18T17:58:23.124321+0000","last_clean":"2023-10-18T17:58:23.124321+0000","last_became_active":"2023-10-18T17:58:23.124019+0000","last_became_peered":"2023-10-18T17:58:23.124019+0000","last_unstale":"2023-10-18T17:58:23.124321+0000","last_undegraded":"2023-10-18T17:58:23.124321+0000","last_fullsized":"2023-10-18T17:58:23.124321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111016+0000","last_change":"2023-10-18T17:58:22.692184+0000","last_active":"2023-10-18T17:58:23.111016+0000","last_peered":"2023-10-18T17:58:23.111016+0000","last_clean":"2023-10-18T17:58:23.111016+0000","last_became_active":"2023-10-18T17:58:22.691802+0000","last_became_peered":"2023-10-18T17:58:22.691802+0000","last_unstale":"2023-10-18T17:58:23.111016+0000","last_undegraded":"2023-10-18T17:58:23.111016+0000","last_fullsized":"2023-10-18T17:58:23.111016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126067+0000","last_change":"2023-10-18T17:58:23.116039+0000","last_active":"2023-10-18T17:58:23.126067+0000","last_peered":"2023-10-18T17:58:23.126067+0000","last_clean":"2023-10-18T17:58:23.126067+0000","last_became_active":"2023-10-18T17:58:23.115715+0000","last_became_peered":"2023-10-18T17:58:23.115715+0000","last_unstale":"2023-10-18T17:58:23.126067+0000","last_undegraded":"2023-10-18T17:58:23.126067+0000","last_fullsized":"2023-10-18T17:58:23.126067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111682+0000","last_change":"2023-10-18T17:58:22.687555+0000","last_active":"2023-10-18T17:58:23.111682+0000","last_peered":"2023-10-18T17:58:23.111682+0000","last_clean":"2023-10-18T17:58:23.111682+0000","last_became_active":"2023-10-18T17:58:22.687269+0000","last_became_peered":"2023-10-18T17:58:22.687269+0000","last_unstale":"2023-10-18T17:58:23.111682+0000","last_undegraded":"2023-10-18T17:58:23.111682+0000","last_fullsized":"2023-10-18T17:58:23.111682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127657+0000","last_change":"2023-10-18T17:58:23.120769+0000","last_active":"2023-10-18T17:58:23.127657+0000","last_peered":"2023-10-18T17:58:23.127657+0000","last_clean":"2023-10-18T17:58:23.127657+0000","last_became_active":"2023-10-18T17:58:23.120522+0000","last_became_peered":"2023-10-18T17:58:23.120522+0000","last_unstale":"2023-10-18T17:58:23.127657+0000","last_undegraded":"2023-10-18T17:58:23.127657+0000","last_fullsized":"2023-10-18T17:58:23.127657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104694+0000","last_change":"2023-10-18T17:58:22.689656+0000","last_active":"2023-10-18T17:58:23.104694+0000","last_peered":"2023-10-18T17:58:23.104694+0000","last_clean":"2023-10-18T17:58:23.104694+0000","last_became_active":"2023-10-18T17:58:22.689376+0000","last_became_peered":"2023-10-18T17:58:22.689376+0000","last_unstale":"2023-10-18T17:58:23.104694+0000","last_undegraded":"2023-10-18T17:58:23.104694+0000","last_fullsized":"2023-10-18T17:58:23.104694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126926+0000","last_change":"2023-10-18T17:58:23.120357+0000","last_active":"2023-10-18T17:58:23.126926+0000","last_peered":"2023-10-18T17:58:23.126926+0000","last_clean":"2023-10-18T17:58:23.126926+0000","last_became_active":"2023-10-18T17:58:23.118878+0000","last_became_peered":"2023-10-18T17:58:23.118878+0000","last_unstale":"2023-10-18T17:58:23.126926+0000","last_undegraded":"2023-10-18T17:58:23.126926+0000","last_fullsized":"2023-10-18T17:58:23.126926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126262+0000","last_change":"2023-10-18T17:58:23.126262+0000","last_active":"2023-10-18T17:58:23.126262+0000","last_peered":"2023-10-18T17:58:23.126262+0000","last_clean":"2023-10-18T17:58:23.126262+0000","last_became_active":"2023-10-18T17:58:23.125988+0000","last_became_peered":"2023-10-18T17:58:23.125988+0000","last_unstale":"2023-10-18T17:58:23.126262+0000","last_undegraded":"2023-10-18T17:58:23.126262+0000","last_fullsized":"2023-10-18T17:58:23.126262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.683881+0000","last_change":"2023-10-18T17:58:22.680967+0000","last_active":"2023-10-18T17:58:22.683881+0000","last_peered":"2023-10-18T17:58:22.683881+0000","last_clean":"2023-10-18T17:58:22.683881+0000","last_became_active":"2023-10-18T17:58:22.679976+0000","last_became_peered":"2023-10-18T17:58:22.679976+0000","last_unstale":"2023-10-18T17:58:22.683881+0000","last_undegraded":"2023-10-18T17:58:22.683881+0000","last_fullsized":"2023-10-18T17:58:22.683881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135616+0000","last_change":"2023-10-18T17:58:23.131014+0000","last_active":"2023-10-18T17:58:23.135616+0000","last_peered":"2023-10-18T17:58:23.135616+0000","last_clean":"2023-10-18T17:58:23.135616+0000","last_became_active":"2023-10-18T17:58:23.130516+0000","last_became_peered":"2023-10-18T17:58:23.130516+0000","last_unstale":"2023-10-18T17:58:23.135616+0000","last_undegraded":"2023-10-18T17:58:23.135616+0000","last_fullsized":"2023-10-18T17:58:23.135616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.109880+0000","last_change":"2023-10-18T17:58:22.684193+0000","last_active":"2023-10-18T17:58:23.109880+0000","last_peered":"2023-10-18T17:58:23.109880+0000","last_clean":"2023-10-18T17:58:23.109880+0000","last_became_active":"2023-10-18T17:58:22.683668+0000","last_became_peered":"2023-10-18T17:58:22.683668+0000","last_unstale":"2023-10-18T17:58:23.109880+0000","last_undegraded":"2023-10-18T17:58:23.109880+0000","last_fullsized":"2023-10-18T17:58:23.109880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128357+0000","last_change":"2023-10-18T17:58:23.123700+0000","last_active":"2023-10-18T17:58:23.128357+0000","last_peered":"2023-10-18T17:58:23.128357+0000","last_clean":"2023-10-18T17:58:23.128357+0000","last_became_active":"2023-10-18T17:58:23.123224+0000","last_became_peered":"2023-10-18T17:58:23.123224+0000","last_unstale":"2023-10-18T17:58:23.128357+0000","last_undegraded":"2023-10-18T17:58:23.128357+0000","last_fullsized":"2023-10-18T17:58:23.128357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116459+0000","last_change":"2023-10-18T17:58:22.674799+0000","last_active":"2023-10-18T17:58:23.116459+0000","last_peered":"2023-10-18T17:58:23.116459+0000","last_clean":"2023-10-18T17:58:23.116459+0000","last_became_active":"2023-10-18T17:58:22.674479+0000","last_became_peered":"2023-10-18T17:58:22.674479+0000","last_unstale":"2023-10-18T17:58:23.116459+0000","last_undegraded":"2023-10-18T17:58:23.116459+0000","last_fullsized":"2023-10-18T17:58:23.116459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127727+0000","last_change":"2023-10-18T17:58:23.121289+0000","last_active":"2023-10-18T17:58:23.127727+0000","last_peered":"2023-10-18T17:58:23.127727+0000","last_clean":"2023-10-18T17:58:23.127727+0000","last_became_active":"2023-10-18T17:58:23.120024+0000","last_became_peered":"2023-10-18T17:58:23.120024+0000","last_unstale":"2023-10-18T17:58:23.127727+0000","last_undegraded":"2023-10-18T17:58:23.127727+0000","last_fullsized":"2023-10-18T17:58:23.127727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110283+0000","last_change":"2023-10-18T17:58:22.675422+0000","last_active":"2023-10-18T17:58:23.110283+0000","last_peered":"2023-10-18T17:58:23.110283+0000","last_clean":"2023-10-18T17:58:23.110283+0000","last_became_active":"2023-10-18T17:58:22.675074+0000","last_became_peered":"2023-10-18T17:58:22.675074+0000","last_unstale":"2023-10-18T17:58:23.110283+0000","last_undegraded":"2023-10-18T17:58:23.110283+0000","last_fullsized":"2023-10-18T17:58:23.110283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125653+0000","last_change":"2023-10-18T17:58:23.125653+0000","last_active":"2023-10-18T17:58:23.125653+0000","last_peered":"2023-10-18T17:58:23.125653+0000","last_clean":"2023-10-18T17:58:23.125653+0000","last_became_active":"2023-10-18T17:58:23.124682+0000","last_became_peered":"2023-10-18T17:58:23.124682+0000","last_unstale":"2023-10-18T17:58:23.125653+0000","last_undegraded":"2023-10-18T17:58:23.125653+0000","last_fullsized":"2023-10-18T17:58:23.125653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115530+0000","last_change":"2023-10-18T17:58:22.673247+0000","last_active":"2023-10-18T17:58:23.115530+0000","last_peered":"2023-10-18T17:58:23.115530+0000","last_clean":"2023-10-18T17:58:23.115530+0000","last_became_active":"2023-10-18T17:58:22.672903+0000","last_became_peered":"2023-10-18T17:58:22.672903+0000","last_unstale":"2023-10-18T17:58:23.115530+0000","last_undegraded":"2023-10-18T17:58:23.115530+0000","last_fullsized":"2023-10-18T17:58:23.115530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129965+0000","last_change":"2023-10-18T17:58:23.125035+0000","last_active":"2023-10-18T17:58:23.129965+0000","last_peered":"2023-10-18T17:58:23.129965+0000","last_clean":"2023-10-18T17:58:23.129965+0000","last_became_active":"2023-10-18T17:58:23.124761+0000","last_became_peered":"2023-10-18T17:58:23.124761+0000","last_unstale":"2023-10-18T17:58:23.129965+0000","last_undegraded":"2023-10-18T17:58:23.129965+0000","last_fullsized":"2023-10-18T17:58:23.129965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.674667+0000","last_change":"2023-10-18T17:58:22.672712+0000","last_active":"2023-10-18T17:58:22.674667+0000","last_peered":"2023-10-18T17:58:22.674667+0000","last_clean":"2023-10-18T17:58:22.674667+0000","last_became_active":"2023-10-18T17:58:22.672368+0000","last_became_peered":"2023-10-18T17:58:22.672368+0000","last_unstale":"2023-10-18T17:58:22.674667+0000","last_undegraded":"2023-10-18T17:58:22.674667+0000","last_fullsized":"2023-10-18T17:58:22.674667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135798+0000","last_change":"2023-10-18T17:58:23.127848+0000","last_active":"2023-10-18T17:58:23.135798+0000","last_peered":"2023-10-18T17:58:23.135798+0000","last_clean":"2023-10-18T17:58:23.135798+0000","last_became_active":"2023-10-18T17:58:23.125611+0000","last_became_peered":"2023-10-18T17:58:23.125611+0000","last_unstale":"2023-10-18T17:58:23.135798+0000","last_undegraded":"2023-10-18T17:58:23.135798+0000","last_fullsized":"2023-10-18T17:58:23.135798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T17:58:22.679422+0000","last_change":"2023-10-18T17:58:22.675355+0000","last_active":"2023-10-18T17:58:22.679422+0000","last_peered":"2023-10-18T17:58:22.679422+0000","last_clean":"2023-10-18T17:58:22.679422+0000","last_became_active":"2023-10-18T17:58:22.675050+0000","last_became_peered":"2023-10-18T17:58:22.675050+0000","last_unstale":"2023-10-18T17:58:22.679422+0000","last_undegraded":"2023-10-18T17:58:22.679422+0000","last_fullsized":"2023-10-18T17:58:22.679422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129429+0000","last_change":"2023-10-18T17:58:23.124889+0000","last_active":"2023-10-18T17:58:23.129429+0000","last_peered":"2023-10-18T17:58:23.129429+0000","last_clean":"2023-10-18T17:58:23.129429+0000","last_became_active":"2023-10-18T17:58:23.124619+0000","last_became_peered":"2023-10-18T17:58:23.124619+0000","last_unstale":"2023-10-18T17:58:23.129429+0000","last_undegraded":"2023-10-18T17:58:23.129429+0000","last_fullsized":"2023-10-18T17:58:23.129429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111694+0000","last_change":"2023-10-18T17:58:22.692437+0000","last_active":"2023-10-18T17:58:23.111694+0000","last_peered":"2023-10-18T17:58:23.111694+0000","last_clean":"2023-10-18T17:58:23.111694+0000","last_became_active":"2023-10-18T17:58:22.692043+0000","last_became_peered":"2023-10-18T17:58:22.692043+0000","last_unstale":"2023-10-18T17:58:23.111694+0000","last_undegraded":"2023-10-18T17:58:23.111694+0000","last_fullsized":"2023-10-18T17:58:23.111694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7769999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.202}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.881}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.387}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2429999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.601}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297292,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074548,"statfs":{"total":96636764160,"available":96332337152,"internally_reserved":0,"allocated":200704,"data_stored":70752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297304,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074536,"statfs":{"total":96636764160,"available":96332324864,"internally_reserved":0,"allocated":212992,"data_stored":77931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T17:58:29.811 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T17:58:29.812 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-18T17:58:29.812 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-18T17:58:29.812 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-18T17:58:29.813 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-18T17:58:30.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.002+0000 7f021161b700 1 Processor -- start 2023-10-18T17:58:30.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f021161b700 1 -- start start 2023-10-18T17:58:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f021161b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f020c0b9b20 0x7f020c065b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f021161b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c153290 0x7f020c15a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f021161b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c0681f0 con 0x7f020c067d90 2023-10-18T17:58:30.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f021161b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0c0cf0 con 0x7f020c0b9b20 2023-10-18T17:58:30.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f021161b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c153720 con 0x7f020c153290 2023-10-18T17:58:30.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f020affd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f020c0b9b20 0x7f020c065b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f020affd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f020c0b9b20 0x7f020c065b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36294/0 (socket says 172.21.15.72:36294) 2023-10-18T17:58:30.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f020affd700 1 -- 172.21.15.72:0/2451134827 learned_addr learned my addr 172.21.15.72:0/2451134827 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:30.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f020a7fc700 1 --1- 172.21.15.72:0/2451134827 >> v1:172.21.15.179:6789/0 conn(0x7f020c067d90 0x7f020c0645a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:33408/0 (socket says 172.21.15.72:33408) 2023-10-18T17:58:30.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.006+0000 7f020b7fe700 1 --2- 172.21.15.72:0/2451134827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c153290 0x7f020c15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f0209ffb700 1 -- 172.21.15.72:0/2451134827 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546152818 0 0) 0x7f020c0681f0 con 0x7f020c067d90 2023-10-18T17:58:30.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f0209ffb700 1 -- 172.21.15.72:0/2451134827 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f40033f0 con 0x7f020c067d90 2023-10-18T17:58:30.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f020affd700 1 -- 172.21.15.72:0/2451134827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c153290 msgr2=0x7f020c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:30.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f020affd700 1 --2- 172.21.15.72:0/2451134827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c153290 0x7f020c15a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f020affd700 1 -- 172.21.15.72:0/2451134827 >> v1:172.21.15.179:6789/0 conn(0x7f020c067d90 legacy=0x7f020c0645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f020affd700 1 -- 172.21.15.72:0/2451134827 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0c2b70 con 0x7f020c0b9b20 2023-10-18T17:58:30.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f020b7fe700 1 --2- 172.21.15.72:0/2451134827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c153290 0x7f020c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:30.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f020affd700 1 --2- 172.21.15.72:0/2451134827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c0b9b20 0x7f020c065b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f01fc006fd0 tx=0x7f01fc0090f0).ready entity=mon.0 client_cookie=b38fee8a0eae5744 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:30.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f0209ffb700 1 -- 172.21.15.72:0/2451134827 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00f510 con 0x7f020c0b9b20 2023-10-18T17:58:30.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f0209ffb700 1 -- 172.21.15.72:0/2451134827 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc00f670 con 0x7f020c0b9b20 2023-10-18T17:58:30.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f0209ffb700 1 -- 172.21.15.72:0/2451134827 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00f9f0 con 0x7f020c0b9b20 2023-10-18T17:58:30.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f021161b700 1 -- 172.21.15.72:0/2451134827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c0b9b20 msgr2=0x7f020c065b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f021161b700 1 --2- 172.21.15.72:0/2451134827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c0b9b20 0x7f020c065b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f01fc006fd0 tx=0x7f01fc0090f0).stop 2023-10-18T17:58:30.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f021161b700 1 -- 172.21.15.72:0/2451134827 shutdown_connections 2023-10-18T17:58:30.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f021161b700 1 --2- 172.21.15.72:0/2451134827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c153290 0x7f020c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f021161b700 1 --2- 172.21.15.72:0/2451134827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c0b9b20 0x7f020c065b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f021161b700 1 -- 172.21.15.72:0/2451134827 >> 172.21.15.72:0/2451134827 conn(0x7f020c069c80 msgr2=0x7f020c06a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:30.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f021161b700 1 -- 172.21.15.72:0/2451134827 shutdown_connections 2023-10-18T17:58:30.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.010+0000 7f021161b700 1 -- 172.21.15.72:0/2451134827 wait complete. 2023-10-18T17:58:30.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f021161b700 1 Processor -- start 2023-10-18T17:58:30.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f021161b700 1 -- start start 2023-10-18T17:58:30.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f021161b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f020c067d90 0x7f020c0bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f021161b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c0b9b20 0x7f020c0bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f021161b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c153290 0x7f020c149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f021161b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1530d0 con 0x7f020c153290 2023-10-18T17:58:30.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f021161b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c152ac0 con 0x7f020c067d90 2023-10-18T17:58:30.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f021161b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0b9f00 con 0x7f020c0b9b20 2023-10-18T17:58:30.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c153290 0x7f020c149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c153290 0x7f020c149420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36302/0 (socket says 172.21.15.72:36302) 2023-10-18T17:58:30.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020b7fe700 1 -- 172.21.15.72:0/3265263075 learned_addr learned my addr 172.21.15.72:0/3265263075 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:30.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020a7fc700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c0b9b20 0x7f020c0bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020affd700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f020c067d90 0x7f020c0bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020b7fe700 1 -- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c0b9b20 msgr2=0x7f020c0bdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020b7fe700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c0b9b20 0x7f020c0bdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020b7fe700 1 -- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f020c067d90 msgr2=0x7f020c0bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020b7fe700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f020c067d90 0x7f020c0bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020b7fe700 1 -- 172.21.15.72:0/3265263075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0204009a20 con 0x7f020c153290 2023-10-18T17:58:30.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.014+0000 7f020b7fe700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c153290 0x7f020c149420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f0204009f20 tx=0x7f020400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:30.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.018+0000 7f01f2ffd700 1 -- 172.21.15.72:0/3265263075 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020400cc00 con 0x7f020c153290 2023-10-18T17:58:30.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.018+0000 7f01f2ffd700 1 -- 172.21.15.72:0/3265263075 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020400f040 con 0x7f020c153290 2023-10-18T17:58:30.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.018+0000 7f01f2ffd700 1 -- 172.21.15.72:0/3265263075 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204013400 con 0x7f020c153290 2023-10-18T17:58:30.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.018+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01fc009370 con 0x7f020c153290 2023-10-18T17:58:30.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.018+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c152d60 con 0x7f020c153290 2023-10-18T17:58:30.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.018+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c0bfbe0 con 0x7f020c153290 2023-10-18T17:58:30.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.018+0000 7f01f2ffd700 1 -- 172.21.15.72:0/3265263075 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f02040137b0 con 0x7f020c153290 2023-10-18T17:58:30.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.022+0000 7f01f2ffd700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f01f805a510 0x7f01f805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.022+0000 7f01f2ffd700 1 -- 172.21.15.72:0/3265263075 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f02040750a0 con 0x7f020c153290 2023-10-18T17:58:30.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.022+0000 7f020affd700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f01f805a510 0x7f01f805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.022+0000 7f020affd700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f01f805a510 0x7f01f805c9d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f01fc009130 tx=0x7f01fc017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:30.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.026+0000 7f01f2ffd700 1 -- 172.21.15.72:0/3265263075 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f0204048820 con 0x7f020c153290 2023-10-18T17:58:30.229 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.226+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f020c046f00 con 0x7f020c153290 2023-10-18T17:58:30.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.234+0000 7f01f2ffd700 1 -- 172.21.15.72:0/3265263075 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+142151 (secure 0 0 0) 0x7f020404be40 con 0x7f020c153290 2023-10-18T17:58:30.238 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:30.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f01f805a510 msgr2=0x7f01f805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f01f805a510 0x7f01f805c9d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f01fc009130 tx=0x7f01fc017000).stop 2023-10-18T17:58:30.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c153290 msgr2=0x7f020c149420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c153290 0x7f020c149420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f0204009f20 tx=0x7f020400b040).stop 2023-10-18T17:58:30.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 shutdown_connections 2023-10-18T17:58:30.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f01f805a510 0x7f01f805c9d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f020c153290 0x7f020c149420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f020c0b9b20 0x7f020c0bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 --2- 172.21.15.72:0/3265263075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f020c067d90 0x7f020c0bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 >> 172.21.15.72:0/3265263075 conn(0x7f020c069c80 msgr2=0x7f020c0c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:30.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 shutdown_connections 2023-10-18T17:58:30.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.238+0000 7f021161b700 1 -- 172.21.15.72:0/3265263075 wait complete. 2023-10-18T17:58:30.265 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":33696},{"type":"v1","addr":"172.21.15.179:6833","nonce":33696}]},"active_addr":"172.21.15.179:6833/33696","active_change":"2023-10-18T17:58:08.448314+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.179:0","nonce":3088742139}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.179:0","nonce":1846400803}]}]} 2023-10-18T17:58:30.269 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-18T17:58:30.269 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-18T17:58:30.270 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:30.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f1440946700 1 Processor -- start 2023-10-18T17:58:30.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f1440946700 1 -- start start 2023-10-18T17:58:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f1440946700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f143c14a460 0x7f143c150b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f1440946700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143c151050 con 0x7f143c066130 2023-10-18T17:58:30.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f1440946700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143c0bd1c0 con 0x7f143c14a080 2023-10-18T17:58:30.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f1440946700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143c0684a0 con 0x7f143c14a460 2023-10-18T17:58:30.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f1439d9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f143c14a460 0x7f143c150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f143ad9d700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f143c066130 0x7f143c0be7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51076/0 (socket says 172.21.15.72:51076) 2023-10-18T17:58:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f143ad9d700 1 -- 172.21.15.72:0/558710934 learned_addr learned my addr 172.21.15.72:0/558710934 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:30.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.434+0000 7f143a59c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f143c14a080 0x7f143c0bcab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:33420/0 (socket says 172.21.15.72:33420) 2023-10-18T17:58:30.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f143959a700 1 -- 172.21.15.72:0/558710934 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039798537 0 0) 0x7f143c0bd1c0 con 0x7f143c14a080 2023-10-18T17:58:30.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1439d9b700 1 -- 172.21.15.72:0/558710934 >> v1:172.21.15.179:6789/0 conn(0x7f143c14a080 legacy=0x7f143c0bcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1439d9b700 1 -- 172.21.15.72:0/558710934 >> v1:172.21.15.72:6789/0 conn(0x7f143c066130 legacy=0x7f143c0be7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1439d9b700 1 -- 172.21.15.72:0/558710934 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c068780 con 0x7f143c14a460 2023-10-18T17:58:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1439d9b700 1 --2- 172.21.15.72:0/558710934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f143c14a460 0x7f143c150b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f1430009760 tx=0x7f1430009060).ready entity=mon.1 client_cookie=91562adb9ad91692 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f143959a700 1 -- 172.21.15.72:0/558710934 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143000f510 con 0x7f143c14a460 2023-10-18T17:58:30.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f143959a700 1 -- 172.21.15.72:0/558710934 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143000f670 con 0x7f143c14a460 2023-10-18T17:58:30.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f143959a700 1 -- 172.21.15.72:0/558710934 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143000f9f0 con 0x7f143c14a460 2023-10-18T17:58:30.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1440946700 1 -- 172.21.15.72:0/558710934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f143c14a460 msgr2=0x7f143c150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1440946700 1 --2- 172.21.15.72:0/558710934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f143c14a460 0x7f143c150b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f1430009760 tx=0x7f1430009060).stop 2023-10-18T17:58:30.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1440946700 1 -- 172.21.15.72:0/558710934 shutdown_connections 2023-10-18T17:58:30.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1440946700 1 --2- 172.21.15.72:0/558710934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f143c14a460 0x7f143c150b60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1440946700 1 -- 172.21.15.72:0/558710934 >> 172.21.15.72:0/558710934 conn(0x7f143c0620d0 msgr2=0x7f143c0624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:30.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1440946700 1 -- 172.21.15.72:0/558710934 shutdown_connections 2023-10-18T17:58:30.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.438+0000 7f1440946700 1 -- 172.21.15.72:0/558710934 wait complete. 2023-10-18T17:58:30.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.442+0000 7f1440946700 1 Processor -- start 2023-10-18T17:58:30.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.442+0000 7f1440946700 1 -- start start 2023-10-18T17:58:30.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f1440946700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c066130 0x7f143c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f1440946700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f143c14a080 0x7f143c14e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f1440946700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f143c14a460 0x7f143c0bb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f1440946700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0b7950 con 0x7f143c066130 2023-10-18T17:58:30.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f1440946700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0bb020 con 0x7f143c14a080 2023-10-18T17:58:30.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f1440946700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c0b6f90 con 0x7f143c14a460 2023-10-18T17:58:30.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c066130 0x7f143c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c066130 0x7f143c15e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36310/0 (socket says 172.21.15.72:36310) 2023-10-18T17:58:30.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143a59c700 1 -- 172.21.15.72:0/2762491987 learned_addr learned my addr 172.21.15.72:0/2762491987 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:30.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f1439d9b700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f143c14a080 0x7f143c14e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143ad9d700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f143c14a460 0x7f143c0bb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143a59c700 1 -- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f143c14a460 msgr2=0x7f143c0bb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143a59c700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f143c14a460 0x7f143c0bb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143a59c700 1 -- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f143c14a080 msgr2=0x7f143c14e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143a59c700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f143c14a080 0x7f143c14e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143a59c700 1 -- 172.21.15.72:0/2762491987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14300092a0 con 0x7f143c066130 2023-10-18T17:58:30.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f143a59c700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c066130 0x7f143c15e750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f142c00c4c0 tx=0x7f142c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:30.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f142a7fc700 1 -- 172.21.15.72:0/2762491987 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c007170 con 0x7f143c066130 2023-10-18T17:58:30.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f142a7fc700 1 -- 172.21.15.72:0/2762491987 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c0072d0 con 0x7f143c066130 2023-10-18T17:58:30.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143c15ed30 con 0x7f143c066130 2023-10-18T17:58:30.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f142a7fc700 1 -- 172.21.15.72:0/2762491987 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0046b0 con 0x7f143c066130 2023-10-18T17:58:30.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.446+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143c15f2b0 con 0x7f143c066130 2023-10-18T17:58:30.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.450+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143c0bf530 con 0x7f143c066130 2023-10-18T17:58:30.455 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.450+0000 7f142a7fc700 1 -- 172.21.15.72:0/2762491987 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f142c004a60 con 0x7f143c066130 2023-10-18T17:58:30.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.450+0000 7f142a7fc700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f141c05a510 0x7f141c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.454+0000 7f142a7fc700 1 -- 172.21.15.72:0/2762491987 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f142c073f00 con 0x7f143c066130 2023-10-18T17:58:30.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.454+0000 7f1439d9b700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f141c05a510 0x7f141c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.454+0000 7f1439d9b700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f141c05a510 0x7f141c05c9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f143c0bc2d0 tx=0x7f1430017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:30.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.462+0000 7f142a7fc700 1 -- 172.21.15.72:0/2762491987 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f142c047680 con 0x7f143c066130 2023-10-18T17:58:30.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.634+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f143c06c190 con 0x7f143c066130 2023-10-18T17:58:30.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.634+0000 7f142a7fc700 1 -- 172.21.15.72:0/2762491987 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f142c04aca0 con 0x7f143c066130 2023-10-18T17:58:30.640 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:30.640 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":22,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:24.095038+0000","last_up_change":"2023-10-18T17:58:14.550597+0000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T17:58:13.627096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T17:58:15.732786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T17:58:21.178314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T17:58:22.112742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":33232},{"type":"v1","addr":"172.21.15.72:6810","nonce":33232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6813","nonce":33232},{"type":"v1","addr":"172.21.15.72:6815","nonce":33232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":33232},{"type":"v1","addr":"172.21.15.72:6826","nonce":33232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":33232},{"type":"v1","addr":"172.21.15.72:6820","nonce":33232}]},"public_addr":"172.21.15.72:6810/33232","cluster_addr":"172.21.15.72:6815/33232","heartbeat_back_addr":"172.21.15.72:6826/33232","heartbeat_front_addr":"172.21.15.72:6820/33232","state":["exists","up"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6809","nonce":33229},{"type":"v1","addr":"172.21.15.72:6812","nonce":33229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":33229},{"type":"v1","addr":"172.21.15.72:6819","nonce":33229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":33229},{"type":"v1","addr":"172.21.15.72:6831","nonce":33229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":33229},{"type":"v1","addr":"172.21.15.72:6825","nonce":33229}]},"public_addr":"172.21.15.72:6812/33229","cluster_addr":"172.21.15.72:6819/33229","heartbeat_back_addr":"172.21.15.72:6831/33229","heartbeat_front_addr":"172.21.15.72:6825/33229","state":["exists","up"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":33231},{"type":"v1","addr":"172.21.15.72:6801","nonce":33231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":33231},{"type":"v1","addr":"172.21.15.72:6803","nonce":33231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":33231},{"type":"v1","addr":"172.21.15.72:6807","nonce":33231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":33231},{"type":"v1","addr":"172.21.15.72:6805","nonce":33231}]},"public_addr":"172.21.15.72:6801/33231","cluster_addr":"172.21.15.72:6803/33231","heartbeat_back_addr":"172.21.15.72:6807/33231","heartbeat_front_addr":"172.21.15.72:6805/33231","state":["exists","up"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":33228},{"type":"v1","addr":"172.21.15.72:6814","nonce":33228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":33228},{"type":"v1","addr":"172.21.15.72:6821","nonce":33228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":33228},{"type":"v1","addr":"172.21.15.72:6830","nonce":33228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":33228},{"type":"v1","addr":"172.21.15.72:6827","nonce":33228}]},"public_addr":"172.21.15.72:6814/33228","cluster_addr":"172.21.15.72:6821/33228","heartbeat_back_addr":"172.21.15.72:6830/33228","heartbeat_front_addr":"172.21.15.72:6827/33228","state":["exists","up"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":34034},{"type":"v1","addr":"172.21.15.179:6809","nonce":34034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":34034},{"type":"v1","addr":"172.21.15.179:6813","nonce":34034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":34034},{"type":"v1","addr":"172.21.15.179:6821","nonce":34034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":34034},{"type":"v1","addr":"172.21.15.179:6817","nonce":34034}]},"public_addr":"172.21.15.179:6809/34034","cluster_addr":"172.21.15.179:6813/34034","heartbeat_back_addr":"172.21.15.179:6821/34034","heartbeat_front_addr":"172.21.15.179:6817/34034","state":["exists","up"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":34044},{"type":"v1","addr":"172.21.15.179:6801","nonce":34044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":34044},{"type":"v1","addr":"172.21.15.179:6803","nonce":34044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":34044},{"type":"v1","addr":"172.21.15.179:6807","nonce":34044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":34044},{"type":"v1","addr":"172.21.15.179:6805","nonce":34044}]},"public_addr":"172.21.15.179:6801/34044","cluster_addr":"172.21.15.179:6803/34044","heartbeat_back_addr":"172.21.15.179:6807/34044","heartbeat_front_addr":"172.21.15.179:6805/34044","state":["exists","up"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":34036},{"type":"v1","addr":"172.21.15.179:6825","nonce":34036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":34036},{"type":"v1","addr":"172.21.15.179:6827","nonce":34036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":34036},{"type":"v1","addr":"172.21.15.179:6831","nonce":34036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":34036},{"type":"v1","addr":"172.21.15.179:6829","nonce":34036}]},"public_addr":"172.21.15.179:6825/34036","cluster_addr":"172.21.15.179:6827/34036","heartbeat_back_addr":"172.21.15.179:6831/34036","heartbeat_front_addr":"172.21.15.179:6829/34036","state":["exists","up"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":34035},{"type":"v1","addr":"172.21.15.179:6812","nonce":34035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":34035},{"type":"v1","addr":"172.21.15.179:6816","nonce":34035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":34035},{"type":"v1","addr":"172.21.15.179:6823","nonce":34035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":34035},{"type":"v1","addr":"172.21.15.179:6820","nonce":34035}]},"public_addr":"172.21.15.179:6812/34035","cluster_addr":"172.21.15.179:6816/34035","heartbeat_back_addr":"172.21.15.179:6823/34035","heartbeat_front_addr":"172.21.15.179:6820/34035","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.697876+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.661119+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.579136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.792036+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.069244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:11.971339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.176699+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.197004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:30.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f141c05a510 msgr2=0x7f141c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f141c05a510 0x7f141c05c9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f143c0bc2d0 tx=0x7f1430017000).stop 2023-10-18T17:58:30.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c066130 msgr2=0x7f143c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c066130 0x7f143c15e750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f142c00c4c0 tx=0x7f142c007c10).stop 2023-10-18T17:58:30.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 shutdown_connections 2023-10-18T17:58:30.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f141c05a510 0x7f141c05c9d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f143c14a460 0x7f143c0bb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f143c14a080 0x7f143c14e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 --2- 172.21.15.72:0/2762491987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f143c066130 0x7f143c15e750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.638+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 >> 172.21.15.72:0/2762491987 conn(0x7f143c0620d0 msgr2=0x7f143c0624b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:30.646 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.642+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 shutdown_connections 2023-10-18T17:58:30.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.642+0000 7f1440946700 1 -- 172.21.15.72:0/2762491987 wait complete. 2023-10-18T17:58:30.664 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-18T17:58:30.665 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T17:58:30.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.830+0000 7f95b1307700 1 Processor -- start 2023-10-18T17:58:30.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.830+0000 7f95b1307700 1 -- start start 2023-10-18T17:58:30.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95b1307700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac0bf340 0x7f95ac0bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95b1307700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f95ac14e3d0 0x7f95ac14e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95b1307700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac14dc30 con 0x7f95ac066180 2023-10-18T17:58:30.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95b1307700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0bbb60 con 0x7f95ac14e3d0 2023-10-18T17:58:30.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95b1307700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0bbe00 con 0x7f95ac0bf340 2023-10-18T17:58:30.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95abfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac0bf340 0x7f95ac0bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95abfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac0bf340 0x7f95ac0bf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59162/0 (socket says 172.21.15.72:59162) 2023-10-18T17:58:30.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95abfff700 1 -- 172.21.15.72:0/2973146507 learned_addr learned my addr 172.21.15.72:0/2973146507 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:30.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95a3fff700 1 --2- 172.21.15.72:0/2973146507 >> v2:172.21.15.72:3300/0 conn(0x7f95ac14e3d0 0x7f95ac14e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95abfff700 1 -- 172.21.15.72:0/2973146507 >> v2:172.21.15.72:3300/0 conn(0x7f95ac14e3d0 msgr2=0x7f95ac14e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:30.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95abfff700 1 --2- 172.21.15.72:0/2973146507 >> v2:172.21.15.72:3300/0 conn(0x7f95ac14e3d0 0x7f95ac14e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95abfff700 1 -- 172.21.15.72:0/2973146507 >> v1:172.21.15.72:6789/0 conn(0x7f95ac066180 legacy=0x7f95ac151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95abfff700 1 -- 172.21.15.72:0/2973146507 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0bc0c0 con 0x7f95ac0bf340 2023-10-18T17:58:30.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.834+0000 7f95abfff700 1 --2- 172.21.15.72:0/2973146507 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac0bf340 0x7f95ac0bf720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f959c006fd0 tx=0x7f959c0090f0).ready entity=mon.2 client_cookie=1edbd77cead1a270 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:30.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95ab7fe700 1 -- 172.21.15.72:0/2973146507 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00b610 con 0x7f95ac0bf340 2023-10-18T17:58:30.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95ab7fe700 1 -- 172.21.15.72:0/2973146507 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00b770 con 0x7f95ac0bf340 2023-10-18T17:58:30.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95ab7fe700 1 -- 172.21.15.72:0/2973146507 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00baf0 con 0x7f95ac0bf340 2023-10-18T17:58:30.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95b1307700 1 -- 172.21.15.72:0/2973146507 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac0bf340 msgr2=0x7f95ac0bf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95b1307700 1 --2- 172.21.15.72:0/2973146507 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac0bf340 0x7f95ac0bf720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f959c006fd0 tx=0x7f959c0090f0).stop 2023-10-18T17:58:30.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95b1307700 1 -- 172.21.15.72:0/2973146507 shutdown_connections 2023-10-18T17:58:30.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95b1307700 1 --2- 172.21.15.72:0/2973146507 >> v2:172.21.15.72:3300/0 conn(0x7f95ac14e3d0 0x7f95ac14e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95b1307700 1 --2- 172.21.15.72:0/2973146507 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac0bf340 0x7f95ac0bf720 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95b1307700 1 -- 172.21.15.72:0/2973146507 >> 172.21.15.72:0/2973146507 conn(0x7f95ac05ddc0 msgr2=0x7f95ac05e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:30.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95b1307700 1 -- 172.21.15.72:0/2973146507 shutdown_connections 2023-10-18T17:58:30.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.838+0000 7f95b1307700 1 -- 172.21.15.72:0/2973146507 wait complete. 2023-10-18T17:58:30.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95b1307700 1 Processor -- start 2023-10-18T17:58:30.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95b1307700 1 -- start start 2023-10-18T17:58:30.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95b1307700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac066180 0x7f95ac151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95b1307700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95ac0bf340 0x7f95ac152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95b1307700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac14e3d0 0x7f95ac153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95b1307700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac164700 con 0x7f95ac0bf340 2023-10-18T17:58:30.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95b1307700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1537a0 con 0x7f95ac066180 2023-10-18T17:58:30.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95b1307700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac164420 con 0x7f95ac14e3d0 2023-10-18T17:58:30.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95a3fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95ac0bf340 0x7f95ac152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95a3fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95ac0bf340 0x7f95ac152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36320/0 (socket says 172.21.15.72:36320) 2023-10-18T17:58:30.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95a3fff700 1 -- 172.21.15.72:0/3910869843 learned_addr learned my addr 172.21.15.72:0/3910869843 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:30.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95abfff700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac066180 0x7f95ac151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.842+0000 7f95b0b06700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac14e3d0 0x7f95ac153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a3fff700 1 -- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac14e3d0 msgr2=0x7f95ac153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a3fff700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac14e3d0 0x7f95ac153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a3fff700 1 -- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac066180 msgr2=0x7f95ac151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:30.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a3fff700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac066180 0x7f95ac151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:30.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a3fff700 1 -- 172.21.15.72:0/3910869843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c009370 con 0x7f95ac0bf340 2023-10-18T17:58:30.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a3fff700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95ac0bf340 0x7f95ac152ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f959400e8d0 tx=0x7f9594009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:30.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a8ff9700 1 -- 172.21.15.72:0/3910869843 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594012600 con 0x7f95ac0bf340 2023-10-18T17:58:30.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a8ff9700 1 -- 172.21.15.72:0/3910869843 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9594012760 con 0x7f95ac0bf340 2023-10-18T17:58:30.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac149d90 con 0x7f95ac0bf340 2023-10-18T17:58:30.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a8ff9700 1 -- 172.21.15.72:0/3910869843 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594012ae0 con 0x7f95ac0bf340 2023-10-18T17:58:30.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac163dd0 con 0x7f95ac0bf340 2023-10-18T17:58:30.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac046f00 con 0x7f95ac0bf340 2023-10-18T17:58:30.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.846+0000 7f95a8ff9700 1 -- 172.21.15.72:0/3910869843 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f959401b400 con 0x7f95ac0bf340 2023-10-18T17:58:30.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.850+0000 7f95a8ff9700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f958c05a510 0x7f958c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:30.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.850+0000 7f95a8ff9700 1 -- 172.21.15.72:0/3910869843 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9594074b50 con 0x7f95ac0bf340 2023-10-18T17:58:30.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.850+0000 7f95abfff700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f958c05a510 0x7f958c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:30.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.850+0000 7f95abfff700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f958c05a510 0x7f958c05c9d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f959c009fd0 tx=0x7f959c017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:30.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:30.854+0000 7f95a8ff9700 1 -- 172.21.15.72:0/3910869843 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f9594048210 con 0x7f95ac0bf340 2023-10-18T17:58:31.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.014+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95ac164560 con 0x7f95ac0bf340 2023-10-18T17:58:31.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.014+0000 7f95a8ff9700 1 -- 172.21.15.72:0/3910869843 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f95ac164560 con 0x7f95ac0bf340 2023-10-18T17:58:31.020 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:31.020 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":22,"fsid":"42f1988f-3a5a-4222-b2d6-b1152a417ca8","created":"2023-10-18T17:58:04.559422+0000","modified":"2023-10-18T17:58:24.095038+0000","last_up_change":"2023-10-18T17:58:14.550597+0000","last_in_change":"2023-10-18T17:58:06.744705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-10-18T17:58:13.627096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-18T17:58:15.732786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-18T17:58:21.178314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-18T17:58:22.112742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f23d4e3d-4f47-4c2b-a28d-4997475203b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":33232},{"type":"v1","addr":"172.21.15.72:6810","nonce":33232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6813","nonce":33232},{"type":"v1","addr":"172.21.15.72:6815","nonce":33232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6823","nonce":33232},{"type":"v1","addr":"172.21.15.72:6826","nonce":33232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6817","nonce":33232},{"type":"v1","addr":"172.21.15.72:6820","nonce":33232}]},"public_addr":"172.21.15.72:6810/33232","cluster_addr":"172.21.15.72:6815/33232","heartbeat_back_addr":"172.21.15.72:6826/33232","heartbeat_front_addr":"172.21.15.72:6820/33232","state":["exists","up"]},{"osd":1,"uuid":"214efe82-98a1-4870-9bcb-e3d6d3e42a28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6809","nonce":33229},{"type":"v1","addr":"172.21.15.72:6812","nonce":33229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":33229},{"type":"v1","addr":"172.21.15.72:6819","nonce":33229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":33229},{"type":"v1","addr":"172.21.15.72:6831","nonce":33229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":33229},{"type":"v1","addr":"172.21.15.72:6825","nonce":33229}]},"public_addr":"172.21.15.72:6812/33229","cluster_addr":"172.21.15.72:6819/33229","heartbeat_back_addr":"172.21.15.72:6831/33229","heartbeat_front_addr":"172.21.15.72:6825/33229","state":["exists","up"]},{"osd":2,"uuid":"e8b09618-8597-4bd0-a268-8e3f42c4544f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":33231},{"type":"v1","addr":"172.21.15.72:6801","nonce":33231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":33231},{"type":"v1","addr":"172.21.15.72:6803","nonce":33231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":33231},{"type":"v1","addr":"172.21.15.72:6807","nonce":33231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":33231},{"type":"v1","addr":"172.21.15.72:6805","nonce":33231}]},"public_addr":"172.21.15.72:6801/33231","cluster_addr":"172.21.15.72:6803/33231","heartbeat_back_addr":"172.21.15.72:6807/33231","heartbeat_front_addr":"172.21.15.72:6805/33231","state":["exists","up"]},{"osd":3,"uuid":"711d83d6-b781-4ea7-b2e3-03fbf6f8c73c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6811","nonce":33228},{"type":"v1","addr":"172.21.15.72:6814","nonce":33228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":33228},{"type":"v1","addr":"172.21.15.72:6821","nonce":33228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":33228},{"type":"v1","addr":"172.21.15.72:6830","nonce":33228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":33228},{"type":"v1","addr":"172.21.15.72:6827","nonce":33228}]},"public_addr":"172.21.15.72:6814/33228","cluster_addr":"172.21.15.72:6821/33228","heartbeat_back_addr":"172.21.15.72:6830/33228","heartbeat_front_addr":"172.21.15.72:6827/33228","state":["exists","up"]},{"osd":4,"uuid":"e8d8ca6f-391c-41d1-bc23-5a2ff9b57d39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":34034},{"type":"v1","addr":"172.21.15.179:6809","nonce":34034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6811","nonce":34034},{"type":"v1","addr":"172.21.15.179:6813","nonce":34034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6819","nonce":34034},{"type":"v1","addr":"172.21.15.179:6821","nonce":34034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6815","nonce":34034},{"type":"v1","addr":"172.21.15.179:6817","nonce":34034}]},"public_addr":"172.21.15.179:6809/34034","cluster_addr":"172.21.15.179:6813/34034","heartbeat_back_addr":"172.21.15.179:6821/34034","heartbeat_front_addr":"172.21.15.179:6817/34034","state":["exists","up"]},{"osd":5,"uuid":"1557d884-e99b-4bf1-bed2-77d090311c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":34044},{"type":"v1","addr":"172.21.15.179:6801","nonce":34044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":34044},{"type":"v1","addr":"172.21.15.179:6803","nonce":34044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":34044},{"type":"v1","addr":"172.21.15.179:6807","nonce":34044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":34044},{"type":"v1","addr":"172.21.15.179:6805","nonce":34044}]},"public_addr":"172.21.15.179:6801/34044","cluster_addr":"172.21.15.179:6803/34044","heartbeat_back_addr":"172.21.15.179:6807/34044","heartbeat_front_addr":"172.21.15.179:6805/34044","state":["exists","up"]},{"osd":6,"uuid":"1425becf-644f-4f65-8172-6e39bf4dc94c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":34036},{"type":"v1","addr":"172.21.15.179:6825","nonce":34036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":34036},{"type":"v1","addr":"172.21.15.179:6827","nonce":34036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":34036},{"type":"v1","addr":"172.21.15.179:6831","nonce":34036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":34036},{"type":"v1","addr":"172.21.15.179:6829","nonce":34036}]},"public_addr":"172.21.15.179:6825/34036","cluster_addr":"172.21.15.179:6827/34036","heartbeat_back_addr":"172.21.15.179:6831/34036","heartbeat_front_addr":"172.21.15.179:6829/34036","state":["exists","up"]},{"osd":7,"uuid":"8fdb41ca-c639-4d21-9338-d9416124f970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":34035},{"type":"v1","addr":"172.21.15.179:6812","nonce":34035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":34035},{"type":"v1","addr":"172.21.15.179:6816","nonce":34035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":34035},{"type":"v1","addr":"172.21.15.179:6823","nonce":34035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":34035},{"type":"v1","addr":"172.21.15.179:6820","nonce":34035}]},"public_addr":"172.21.15.179:6812/34035","cluster_addr":"172.21.15.179:6816/34035","heartbeat_back_addr":"172.21.15.179:6823/34035","heartbeat_front_addr":"172.21.15.179:6820/34035","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.697876+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.661119+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.579136+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.792036+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.069244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:11.971339+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.176699+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T17:58:12.197004+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T17:58:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f958c05a510 msgr2=0x7f958c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f958c05a510 0x7f958c05c9d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f959c009fd0 tx=0x7f959c017000).stop 2023-10-18T17:58:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95ac0bf340 msgr2=0x7f95ac152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95ac0bf340 0x7f95ac152ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f959400e8d0 tx=0x7f9594009060).stop 2023-10-18T17:58:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 shutdown_connections 2023-10-18T17:58:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f958c05a510 0x7f958c05c9d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.023 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f95ac14e3d0 0x7f95ac153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95ac0bf340 0x7f95ac152ca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 --2- 172.21.15.72:0/3910869843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95ac066180 0x7f95ac151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 >> 172.21.15.72:0/3910869843 conn(0x7f95ac05ddc0 msgr2=0x7f95ac14c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:31.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 shutdown_connections 2023-10-18T17:58:31.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.018+0000 7f95b1307700 1 -- 172.21.15.72:0/3910869843 wait complete. 2023-10-18T17:58:31.046 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-18T17:58:31.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9dbb45b700 1 Processor -- start 2023-10-18T17:58:31.205 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9dbb45b700 1 -- start start 2023-10-18T17:58:31.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9dbb45b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9db4150420 0x7f9db41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9dbb45b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db414ebe0 con 0x7f9db40c29e0 2023-10-18T17:58:31.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9dbb45b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40bbb40 con 0x7f9db40c2600 2023-10-18T17:58:31.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9dbb45b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db40bbe00 con 0x7f9db4150420 2023-10-18T17:58:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9db99f8700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9db4150420 0x7f9db41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9db91f7700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f9db40c2600 0x7f9db414e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:33434/0 (socket says 172.21.15.72:33434) 2023-10-18T17:58:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9db91f7700 1 -- 172.21.15.72:0/805041552 learned_addr learned my addr 172.21.15.72:0/805041552 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9db99f8700 1 -- 172.21.15.72:0/805041552 >> v1:172.21.15.179:6789/0 conn(0x7f9db40c2600 legacy=0x7f9db414e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9db99f8700 1 -- 172.21.15.72:0/805041552 >> v1:172.21.15.72:6789/0 conn(0x7f9db40c29e0 legacy=0x7f9db4066010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.202+0000 7f9db99f8700 1 -- 172.21.15.72:0/805041552 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40bc0e0 con 0x7f9db4150420 2023-10-18T17:58:31.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9db99f8700 1 --2- 172.21.15.72:0/805041552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db4150420 0x7f9db41508d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f9dac009760 tx=0x7f9dac009060).ready entity=mon.0 client_cookie=a879ab9f7600f49c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dabfff700 1 -- 172.21.15.72:0/805041552 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac00b580 con 0x7f9db4150420 2023-10-18T17:58:31.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dabfff700 1 -- 172.21.15.72:0/805041552 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac00b6e0 con 0x7f9db4150420 2023-10-18T17:58:31.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dabfff700 1 -- 172.21.15.72:0/805041552 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac00ba60 con 0x7f9db4150420 2023-10-18T17:58:31.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dbb45b700 1 -- 172.21.15.72:0/805041552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db4150420 msgr2=0x7f9db41508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dbb45b700 1 --2- 172.21.15.72:0/805041552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db4150420 0x7f9db41508d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f9dac009760 tx=0x7f9dac009060).stop 2023-10-18T17:58:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dbb45b700 1 -- 172.21.15.72:0/805041552 shutdown_connections 2023-10-18T17:58:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dbb45b700 1 --2- 172.21.15.72:0/805041552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db4150420 0x7f9db41508d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.211 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dbb45b700 1 -- 172.21.15.72:0/805041552 >> 172.21.15.72:0/805041552 conn(0x7f9db405dc20 msgr2=0x7f9db405e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:31.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dbb45b700 1 -- 172.21.15.72:0/805041552 shutdown_connections 2023-10-18T17:58:31.212 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.206+0000 7f9dbb45b700 1 -- 172.21.15.72:0/805041552 wait complete. 2023-10-18T17:58:31.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9dbb45b700 1 Processor -- start 2023-10-18T17:58:31.213 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9dbb45b700 1 -- start start 2023-10-18T17:58:31.214 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9dbb45b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db40c2600 0x7f9db40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9dbb45b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9db40c29e0 0x7f9db40bebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9dbb45b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9db4150420 0x7f9db4148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9dbb45b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40b1930 con 0x7f9db40c2600 2023-10-18T17:58:31.215 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9dbb45b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40b0410 con 0x7f9db4150420 2023-10-18T17:58:31.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9dbb45b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db415ef50 con 0x7f9db40c29e0 2023-10-18T17:58:31.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db91f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db40c2600 0x7f9db40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db99f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9db4150420 0x7f9db4148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db89f6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9db40c29e0 0x7f9db40bebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db91f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db40c2600 0x7f9db40aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36332/0 (socket says 172.21.15.72:36332) 2023-10-18T17:58:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db91f7700 1 -- 172.21.15.72:0/3652601447 learned_addr learned my addr 172.21.15.72:0/3652601447 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db91f7700 1 -- 172.21.15.72:0/3652601447 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9db40c29e0 msgr2=0x7f9db40bebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db91f7700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9db40c29e0 0x7f9db40bebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db91f7700 1 -- 172.21.15.72:0/3652601447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9db4150420 msgr2=0x7f9db4148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db91f7700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9db4150420 0x7f9db4148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.210+0000 7f9db91f7700 1 -- 172.21.15.72:0/3652601447 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac0092a0 con 0x7f9db40c2600 2023-10-18T17:58:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.214+0000 7f9db91f7700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db40c2600 0x7f9db40aff20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f9db000c7f0 tx=0x7f9db0001590).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.214+0000 7f9da97fa700 1 -- 172.21.15.72:0/3652601447 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0005d30 con 0x7f9db40c2600 2023-10-18T17:58:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.214+0000 7f9da97fa700 1 -- 172.21.15.72:0/3652601447 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db0005e90 con 0x7f9db40c2600 2023-10-18T17:58:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.214+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db415e3a0 con 0x7f9db40c2600 2023-10-18T17:58:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.214+0000 7f9da97fa700 1 -- 172.21.15.72:0/3652601447 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0007320 con 0x7f9db40c2600 2023-10-18T17:58:31.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.214+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db415e620 con 0x7f9db40c2600 2023-10-18T17:58:31.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.214+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9db406c190 con 0x7f9db40c2600 2023-10-18T17:58:31.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.214+0000 7f9da97fa700 1 -- 172.21.15.72:0/3652601447 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9db0007710 con 0x7f9db40c2600 2023-10-18T17:58:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9da97fa700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9d9805a510 0x7f9d9805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9da97fa700 1 -- 172.21.15.72:0/3652601447 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9db0074730 con 0x7f9db40c2600 2023-10-18T17:58:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9db89f6700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9d9805a510 0x7f9d9805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9da97fa700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.72:6808/33232,v1:172.21.15.72:6810/33232] conn(0x7f9d980638b0 0x7f9d98065d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9da97fa700 1 -- 172.21.15.72:0/3652601447 --> [v2:172.21.15.72:6808/33232,v1:172.21.15.72:6810/33232] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d98066390 con 0x7f9d980638b0 2023-10-18T17:58:31.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9da97fa700 1 -- 172.21.15.72:0/3652601447 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9db0076d50 con 0x7f9db40c2600 2023-10-18T17:58:31.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9db99f8700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.72:6808/33232,v1:172.21.15.72:6810/33232] conn(0x7f9d980638b0 0x7f9d98065d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9db89f6700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9d9805a510 0x7f9d9805c9d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f9db414c060 tx=0x7f9da4007d40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9db99f8700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.72:6808/33232,v1:172.21.15.72:6810/33232] conn(0x7f9d980638b0 0x7f9d98065d10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.218+0000 7f9da97fa700 1 -- 172.21.15.72:0/3652601447 <== osd.0 v2:172.21.15.72:6808/33232 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f9d98066390 con 0x7f9d980638b0 2023-10-18T17:58:31.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 --> [v2:172.21.15.72:6808/33232,v1:172.21.15.72:6810/33232] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9db40b05a0 con 0x7f9d980638b0 2023-10-18T17:58:31.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9da97fa700 1 -- 172.21.15.72:0/3652601447 <== osd.0 v2:172.21.15.72:6808/33232 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9db40b05a0 con 0x7f9d980638b0 2023-10-18T17:58:31.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 >> [v2:172.21.15.72:6808/33232,v1:172.21.15.72:6810/33232] conn(0x7f9d980638b0 msgr2=0x7f9d98065d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.72:6808/33232,v1:172.21.15.72:6810/33232] conn(0x7f9d980638b0 0x7f9d98065d10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9d9805a510 msgr2=0x7f9d9805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9d9805a510 0x7f9d9805c9d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f9db414c060 tx=0x7f9da4007d40).stop 2023-10-18T17:58:31.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db40c2600 msgr2=0x7f9db40aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 --2- 172.21.15.72:0/3652601447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9db40c2600 0x7f9db40aff20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f9db000c7f0 tx=0x7f9db0001590).stop 2023-10-18T17:58:31.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9db99f8700 1 -- 172.21.15.72:0/3652601447 reap_dead start 2023-10-18T17:58:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 shutdown_connections 2023-10-18T17:58:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 >> 172.21.15.72:0/3652601447 conn(0x7f9db405dc20 msgr2=0x7f9db414c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 shutdown_connections 2023-10-18T17:58:31.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.238+0000 7f9dbb45b700 1 -- 172.21.15.72:0/3652601447 wait complete. 2023-10-18T17:58:31.265 INFO:teuthology.orchestra.run.smithi072.stdout:60129542151 2023-10-18T17:58:31.265 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-18T17:58:31.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.422+0000 7f55a5c9b700 1 Processor -- start 2023-10-18T17:58:31.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.422+0000 7f55a5c9b700 1 -- start start 2023-10-18T17:58:31.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f55a01509b0 0x7f55a00621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a0154180 con 0x7f55a00be630 2023-10-18T17:58:31.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a00bfe70 con 0x7f55a00bea10 2023-10-18T17:58:31.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a0068e20 con 0x7f55a01509b0 2023-10-18T17:58:31.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559f7fe700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f55a00be630 0x7f55a00bf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51102/0 (socket says 172.21.15.72:51102) 2023-10-18T17:58:31.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559f7fe700 1 -- 172.21.15.72:0/215507874 learned_addr learned my addr 172.21.15.72:0/215507874 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:31.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559effd700 1 --1- 172.21.15.72:0/215507874 >> v1:172.21.15.179:6789/0 conn(0x7f55a00bea10 0x7f55a00696d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:33444/0 (socket says 172.21.15.72:33444) 2023-10-18T17:58:31.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559ffff700 1 --2- 172.21.15.72:0/215507874 >> v2:172.21.15.72:3300/0 conn(0x7f55a01509b0 0x7f55a00621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559e7fc700 1 -- 172.21.15.72:0/215507874 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138106165 0 0) 0x7f55a0154180 con 0x7f55a00be630 2023-10-18T17:58:31.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559e7fc700 1 -- 172.21.15.72:0/215507874 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55880033f0 con 0x7f55a00be630 2023-10-18T17:58:31.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559e7fc700 1 -- 172.21.15.72:0/215507874 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146265554 0 0) 0x7f55a00bfe70 con 0x7f55a00bea10 2023-10-18T17:58:31.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559e7fc700 1 -- 172.21.15.72:0/215507874 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a0154180 con 0x7f55a00bea10 2023-10-18T17:58:31.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559ffff700 1 -- 172.21.15.72:0/215507874 >> v1:172.21.15.179:6789/0 conn(0x7f55a00bea10 legacy=0x7f55a00696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559ffff700 1 -- 172.21.15.72:0/215507874 >> v1:172.21.15.72:6789/0 conn(0x7f55a00be630 legacy=0x7f55a00bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559e7fc700 1 -- 172.21.15.72:0/215507874 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2458677291 0 0) 0x7f55880033f0 con 0x7f55a00be630 2023-10-18T17:58:31.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559ffff700 1 -- 172.21.15.72:0/215507874 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0153570 con 0x7f55a01509b0 2023-10-18T17:58:31.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559ffff700 1 --2- 172.21.15.72:0/215507874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01509b0 0x7f55a00621d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f5598009e90 tx=0x7f5598009780).ready entity=mon.0 client_cookie=b582fa71fe078254 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559e7fc700 1 -- 172.21.15.72:0/215507874 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800e070 con 0x7f55a01509b0 2023-10-18T17:58:31.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559e7fc700 1 -- 172.21.15.72:0/215507874 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559800b7a0 con 0x7f55a01509b0 2023-10-18T17:58:31.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f559e7fc700 1 -- 172.21.15.72:0/215507874 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590001470 con 0x7f55a01509b0 2023-10-18T17:58:31.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 -- 172.21.15.72:0/215507874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01509b0 msgr2=0x7f55a00621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 --2- 172.21.15.72:0/215507874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01509b0 0x7f55a00621d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f5598009e90 tx=0x7f5598009780).stop 2023-10-18T17:58:31.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 -- 172.21.15.72:0/215507874 shutdown_connections 2023-10-18T17:58:31.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 --2- 172.21.15.72:0/215507874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01509b0 0x7f55a00621d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 -- 172.21.15.72:0/215507874 >> 172.21.15.72:0/215507874 conn(0x7f55a00aeef0 msgr2=0x7f55a00af300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:31.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.426+0000 7f55a5c9b700 1 -- 172.21.15.72:0/215507874 shutdown_connections 2023-10-18T17:58:31.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f55a5c9b700 1 -- 172.21.15.72:0/215507874 wait complete. 2023-10-18T17:58:31.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f55a5c9b700 1 Processor -- start 2023-10-18T17:58:31.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f55a5c9b700 1 -- start start 2023-10-18T17:58:31.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f55a5c9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f55a00be630 0x7f55a00b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f55a5c9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a00bea10 0x7f55a014fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f55a5c9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01509b0 0x7f55a0155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f55a5c9b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a014f6c0 con 0x7f55a01509b0 2023-10-18T17:58:31.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f55a5c9b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00b6860 con 0x7f55a00bea10 2023-10-18T17:58:31.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f55a5c9b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a014deb0 con 0x7f55a00be630 2023-10-18T17:58:31.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f559effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a00bea10 0x7f55a014fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f559effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a00bea10 0x7f55a014fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:56208/0 (socket says 172.21.15.72:56208) 2023-10-18T17:58:31.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f559effd700 1 -- 172.21.15.72:0/2127860195 learned_addr learned my addr 172.21.15.72:0/2127860195 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:31.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.430+0000 7f559f7fe700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f55a00be630 0x7f55a00b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f559ffff700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01509b0 0x7f55a0155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f559effd700 1 -- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f55a00be630 msgr2=0x7f55a00b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f559effd700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f55a00be630 0x7f55a00b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f559effd700 1 -- 172.21.15.72:0/2127860195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01509b0 msgr2=0x7f55a0155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f559effd700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55a01509b0 0x7f55a0155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f559effd700 1 -- 172.21.15.72:0/2127860195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598009a20 con 0x7f55a00bea10 2023-10-18T17:58:31.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f559effd700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a00bea10 0x7f55a014fb50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f559400c4c0 tx=0x7f5594007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f55877fe700 1 -- 172.21.15.72:0/2127860195 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559400ce30 con 0x7f55a00bea10 2023-10-18T17:58:31.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f55877fe700 1 -- 172.21.15.72:0/2127860195 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55940072e0 con 0x7f55a00bea10 2023-10-18T17:58:31.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f55877fe700 1 -- 172.21.15.72:0/2127860195 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55940056e0 con 0x7f55a00bea10 2023-10-18T17:58:31.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a014e190 con 0x7f55a00bea10 2023-10-18T17:58:31.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a0156350 con 0x7f55a00bea10 2023-10-18T17:58:31.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.434+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f55a006c190 con 0x7f55a00bea10 2023-10-18T17:58:31.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.438+0000 7f55877fe700 1 -- 172.21.15.72:0/2127860195 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5594005a90 con 0x7f55a00bea10 2023-10-18T17:58:31.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.438+0000 7f55877fe700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f558c05a540 0x7f558c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.438+0000 7f55877fe700 1 -- 172.21.15.72:0/2127860195 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5594073ee0 con 0x7f55a00bea10 2023-10-18T17:58:31.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.438+0000 7f55877fe700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.72:6809/33229,v1:172.21.15.72:6812/33229] conn(0x7f558c0638e0 0x7f558c065d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.438+0000 7f55877fe700 1 -- 172.21.15.72:0/2127860195 --> [v2:172.21.15.72:6809/33229,v1:172.21.15.72:6812/33229] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f558c0663c0 con 0x7f558c0638e0 2023-10-18T17:58:31.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.438+0000 7f55877fe700 1 -- 172.21.15.72:0/2127860195 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5594076440 con 0x7f55a00bea10 2023-10-18T17:58:31.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.438+0000 7f559f7fe700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f558c05a540 0x7f558c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.438+0000 7f559ffff700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.72:6809/33229,v1:172.21.15.72:6812/33229] conn(0x7f558c0638e0 0x7f558c065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.442+0000 7f559f7fe700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f558c05a540 0x7f558c05ca00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f55900015a0 tx=0x7f5590001a30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.442+0000 7f559ffff700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.72:6809/33229,v1:172.21.15.72:6812/33229] conn(0x7f558c0638e0 0x7f558c065d40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.442+0000 7f55877fe700 1 -- 172.21.15.72:0/2127860195 <== osd.1 v2:172.21.15.72:6809/33229 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f558c0663c0 con 0x7f558c0638e0 2023-10-18T17:58:31.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 --> [v2:172.21.15.72:6809/33229,v1:172.21.15.72:6812/33229] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f55a014f830 con 0x7f558c0638e0 2023-10-18T17:58:31.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55877fe700 1 -- 172.21.15.72:0/2127860195 <== osd.1 v2:172.21.15.72:6809/33229 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f55a014f830 con 0x7f558c0638e0 2023-10-18T17:58:31.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 >> [v2:172.21.15.72:6809/33229,v1:172.21.15.72:6812/33229] conn(0x7f558c0638e0 msgr2=0x7f558c065d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.72:6809/33229,v1:172.21.15.72:6812/33229] conn(0x7f558c0638e0 0x7f558c065d40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f558c05a540 msgr2=0x7f558c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f558c05a540 0x7f558c05ca00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f55900015a0 tx=0x7f5590001a30).stop 2023-10-18T17:58:31.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a00bea10 msgr2=0x7f55a014fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 --2- 172.21.15.72:0/2127860195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a00bea10 0x7f55a014fb50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f559400c4c0 tx=0x7f5594007c10).stop 2023-10-18T17:58:31.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f559ffff700 1 -- 172.21.15.72:0/2127860195 reap_dead start 2023-10-18T17:58:31.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 shutdown_connections 2023-10-18T17:58:31.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 >> 172.21.15.72:0/2127860195 conn(0x7f55a00aeef0 msgr2=0x7f55a00629f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:31.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 shutdown_connections 2023-10-18T17:58:31.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.462+0000 7f55a5c9b700 1 -- 172.21.15.72:0/2127860195 wait complete. 2023-10-18T17:58:31.487 INFO:teuthology.orchestra.run.smithi072.stdout:60129542151 2023-10-18T17:58:31.488 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-18T17:58:31.629 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-18T17:58:31.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.650+0000 7f9692bb9700 1 Processor -- start 2023-10-18T17:58:31.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.650+0000 7f9692bb9700 1 -- start start 2023-10-18T17:58:31.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f968c0bf340 0x7f968c0bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 0x7f968c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c068ea0 con 0x7f968c13d180 2023-10-18T17:58:31.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c154220 con 0x7f968c0bf340 2023-10-18T17:58:31.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c157970 con 0x7f968c0bcaf0 2023-10-18T17:58:31.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9691bb7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f968c0bf340 0x7f968c0bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9691bb7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f968c0bf340 0x7f968c0bf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36366/0 (socket says 172.21.15.72:36366) 2023-10-18T17:58:31.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9691bb7700 1 -- 172.21.15.72:0/4073684479 learned_addr learned my addr 172.21.15.72:0/4073684479 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:31.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f96913b6700 1 --2- 172.21.15.72:0/4073684479 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 0x7f968c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9691bb7700 1 -- 172.21.15.72:0/4073684479 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 msgr2=0x7f968c0689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:31.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9691bb7700 1 --2- 172.21.15.72:0/4073684479 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 0x7f968c0689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9691bb7700 1 -- 172.21.15.72:0/4073684479 >> v1:172.21.15.72:6789/0 conn(0x7f968c13d180 legacy=0x7f968c154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9691bb7700 1 -- 172.21.15.72:0/4073684479 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c157bf0 con 0x7f968c0bf340 2023-10-18T17:58:31.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9691bb7700 1 --2- 172.21.15.72:0/4073684479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f968c0bf340 0x7f968c0bf720 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9688006fd0 tx=0x7f96880090f0).ready entity=mon.0 client_cookie=61facaafa83b25d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9690bb5700 1 -- 172.21.15.72:0/4073684479 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968800b610 con 0x7f968c0bf340 2023-10-18T17:58:31.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9690bb5700 1 -- 172.21.15.72:0/4073684479 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968800b770 con 0x7f968c0bf340 2023-10-18T17:58:31.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9690bb5700 1 -- 172.21.15.72:0/4073684479 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968800baf0 con 0x7f968c0bf340 2023-10-18T17:58:31.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 -- 172.21.15.72:0/4073684479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f968c0bf340 msgr2=0x7f968c0bf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 --2- 172.21.15.72:0/4073684479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f968c0bf340 0x7f968c0bf720 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f9688006fd0 tx=0x7f96880090f0).stop 2023-10-18T17:58:31.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 -- 172.21.15.72:0/4073684479 shutdown_connections 2023-10-18T17:58:31.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 --2- 172.21.15.72:0/4073684479 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 0x7f968c0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 --2- 172.21.15.72:0/4073684479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f968c0bf340 0x7f968c0bf720 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.654+0000 7f9692bb9700 1 -- 172.21.15.72:0/4073684479 >> 172.21.15.72:0/4073684479 conn(0x7f968c05ddc0 msgr2=0x7f968c05e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:31.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.658+0000 7f9692bb9700 1 -- 172.21.15.72:0/4073684479 shutdown_connections 2023-10-18T17:58:31.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.658+0000 7f9692bb9700 1 -- 172.21.15.72:0/4073684479 wait complete. 2023-10-18T17:58:31.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.658+0000 7f9692bb9700 1 Processor -- start 2023-10-18T17:58:31.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.658+0000 7f9692bb9700 1 -- start start 2023-10-18T17:58:31.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.658+0000 7f9692bb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 0x7f968c0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9692bb9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f968c0bf340 0x7f968c0c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9692bb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f968c13d180 0x7f968c152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9692bb9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0bb6f0 con 0x7f968c0bf340 2023-10-18T17:58:31.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9692bb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1525c0 con 0x7f968c13d180 2023-10-18T17:58:31.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9692bb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0bc290 con 0x7f968c0bcaf0 2023-10-18T17:58:31.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f96913b6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f968c0bf340 0x7f968c0c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9691bb7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 0x7f968c0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9691bb7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 0x7f968c0b8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59226/0 (socket says 172.21.15.72:59226) 2023-10-18T17:58:31.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9691bb7700 1 -- 172.21.15.72:0/4217749207 learned_addr learned my addr 172.21.15.72:0/4217749207 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:31.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f96923b8700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f968c13d180 0x7f968c152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f96913b6700 1 -- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 msgr2=0x7f968c0b8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f96913b6700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f968c0bcaf0 0x7f968c0b8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f96913b6700 1 -- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f968c13d180 msgr2=0x7f968c152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f96913b6700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f968c13d180 0x7f968c152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f96913b6700 1 -- 172.21.15.72:0/4217749207 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688009370 con 0x7f968c0bf340 2023-10-18T17:58:31.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f96913b6700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f968c0bf340 0x7f968c0c25d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f967c00e880 tx=0x7f967c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9681ffb700 1 -- 172.21.15.72:0/4217749207 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c012600 con 0x7f968c0bf340 2023-10-18T17:58:31.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9681ffb700 1 -- 172.21.15.72:0/4217749207 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c012760 con 0x7f968c0bf340 2023-10-18T17:58:31.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9681ffb700 1 -- 172.21.15.72:0/4217749207 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c012ae0 con 0x7f968c0bf340 2023-10-18T17:58:31.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c153d40 con 0x7f968c0bf340 2023-10-18T17:58:31.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.662+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c0bc500 con 0x7f968c0bf340 2023-10-18T17:58:31.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f968c046f00 con 0x7f968c0bf340 2023-10-18T17:58:31.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f9681ffb700 1 -- 172.21.15.72:0/4217749207 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f967c01b400 con 0x7f968c0bf340 2023-10-18T17:58:31.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f9681ffb700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f967005a510 0x7f967005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f9681ffb700 1 -- 172.21.15.72:0/4217749207 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f967c074bf0 con 0x7f968c0bf340 2023-10-18T17:58:31.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f9691bb7700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f967005a510 0x7f967005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f9681ffb700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.72:6800/33231,v1:172.21.15.72:6801/33231] conn(0x7f96700638b0 0x7f9670065d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f9681ffb700 1 -- 172.21.15.72:0/4217749207 --> [v2:172.21.15.72:6800/33231,v1:172.21.15.72:6801/33231] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9670066390 con 0x7f96700638b0 2023-10-18T17:58:31.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f9681ffb700 1 -- 172.21.15.72:0/4217749207 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f967c077210 con 0x7f968c0bf340 2023-10-18T17:58:31.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f96923b8700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.72:6800/33231,v1:172.21.15.72:6801/33231] conn(0x7f96700638b0 0x7f9670065d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f9691bb7700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f967005a510 0x7f967005c9d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f968800c010 tx=0x7f9688017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.666+0000 7f96923b8700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.72:6800/33231,v1:172.21.15.72:6801/33231] conn(0x7f96700638b0 0x7f9670065d10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.670+0000 7f9681ffb700 1 -- 172.21.15.72:0/4217749207 <== osd.2 v2:172.21.15.72:6800/33231 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f9670066390 con 0x7f96700638b0 2023-10-18T17:58:31.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 --> [v2:172.21.15.72:6800/33231,v1:172.21.15.72:6801/33231] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f968c157ed0 con 0x7f96700638b0 2023-10-18T17:58:31.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9681ffb700 1 -- 172.21.15.72:0/4217749207 <== osd.2 v2:172.21.15.72:6800/33231 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f968c157ed0 con 0x7f96700638b0 2023-10-18T17:58:31.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 >> [v2:172.21.15.72:6800/33231,v1:172.21.15.72:6801/33231] conn(0x7f96700638b0 msgr2=0x7f9670065d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.72:6800/33231,v1:172.21.15.72:6801/33231] conn(0x7f96700638b0 0x7f9670065d10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f967005a510 msgr2=0x7f967005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f967005a510 0x7f967005c9d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f968800c010 tx=0x7f9688017000).stop 2023-10-18T17:58:31.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f968c0bf340 msgr2=0x7f968c0c25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 --2- 172.21.15.72:0/4217749207 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f968c0bf340 0x7f968c0c25d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f967c00e880 tx=0x7f967c009060).stop 2023-10-18T17:58:31.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f96923b8700 1 -- 172.21.15.72:0/4217749207 reap_dead start 2023-10-18T17:58:31.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 shutdown_connections 2023-10-18T17:58:31.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 >> 172.21.15.72:0/4217749207 conn(0x7f968c05ddc0 msgr2=0x7f968c13be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:31.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.686+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 shutdown_connections 2023-10-18T17:58:31.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.690+0000 7f9692bb9700 1 -- 172.21.15.72:0/4217749207 wait complete. 2023-10-18T17:58:31.716 INFO:teuthology.orchestra.run.smithi072.stdout:60129542150 2023-10-18T17:58:31.717 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-18T17:58:31.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.878+0000 7f897c89b700 1 Processor -- start 2023-10-18T17:58:31.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.878+0000 7f897c89b700 1 -- start start 2023-10-18T17:58:31.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f897c89b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f89781578f0 0x7f89780c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f897c89b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8978151870 0x7f89780bb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f897c89b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8978155b60 con 0x7f89780c2460 2023-10-18T17:58:31.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f897c89b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780bcd80 con 0x7f8978151870 2023-10-18T17:58:31.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f897c89b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89780c2d30 con 0x7f89781578f0 2023-10-18T17:58:31.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f8975d9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f89781578f0 0x7f89780c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f8975d9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f89781578f0 0x7f89780c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59236/0 (socket says 172.21.15.72:59236) 2023-10-18T17:58:31.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f8975d9b700 1 -- 172.21.15.72:0/4190669477 learned_addr learned my addr 172.21.15.72:0/4190669477 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:31.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f8976d9d700 1 --2- 172.21.15.72:0/4190669477 >> v2:172.21.15.72:3300/0 conn(0x7f8978151870 0x7f89780bb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f8975d9b700 1 -- 172.21.15.72:0/4190669477 >> v2:172.21.15.72:3300/0 conn(0x7f8978151870 msgr2=0x7f89780bb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:31.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f8975d9b700 1 --2- 172.21.15.72:0/4190669477 >> v2:172.21.15.72:3300/0 conn(0x7f8978151870 0x7f89780bb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f8975d9b700 1 -- 172.21.15.72:0/4190669477 >> v1:172.21.15.179:6789/0 conn(0x7f89780c2460 legacy=0x7f8978151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f8975d9b700 1 -- 172.21.15.72:0/4190669477 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89781546a0 con 0x7f89781578f0 2023-10-18T17:58:31.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f8975d9b700 1 --2- 172.21.15.72:0/4190669477 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f89781578f0 0x7f89780c2840 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f896c006fd0 tx=0x7f896c0090f0).ready entity=mon.2 client_cookie=68126b1902227339 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f897559a700 1 -- 172.21.15.72:0/4190669477 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00b580 con 0x7f89781578f0 2023-10-18T17:58:31.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f897559a700 1 -- 172.21.15.72:0/4190669477 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c00b6e0 con 0x7f89781578f0 2023-10-18T17:58:31.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.882+0000 7f897559a700 1 -- 172.21.15.72:0/4190669477 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00ba60 con 0x7f89781578f0 2023-10-18T17:58:31.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.886+0000 7f897c89b700 1 -- 172.21.15.72:0/4190669477 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f89781578f0 msgr2=0x7f89780c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.886+0000 7f897c89b700 1 --2- 172.21.15.72:0/4190669477 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f89781578f0 0x7f89780c2840 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f896c006fd0 tx=0x7f896c0090f0).stop 2023-10-18T17:58:31.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.886+0000 7f897c89b700 1 -- 172.21.15.72:0/4190669477 shutdown_connections 2023-10-18T17:58:31.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.886+0000 7f897c89b700 1 --2- 172.21.15.72:0/4190669477 >> v2:172.21.15.72:3300/0 conn(0x7f8978151870 0x7f89780bb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.892 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.886+0000 7f897c89b700 1 --2- 172.21.15.72:0/4190669477 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f89781578f0 0x7f89780c2840 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.892 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.886+0000 7f897c89b700 1 -- 172.21.15.72:0/4190669477 >> 172.21.15.72:0/4190669477 conn(0x7f897805dc20 msgr2=0x7f897805e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:31.892 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.886+0000 7f897c89b700 1 -- 172.21.15.72:0/4190669477 shutdown_connections 2023-10-18T17:58:31.892 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.886+0000 7f897c89b700 1 -- 172.21.15.72:0/4190669477 wait complete. 2023-10-18T17:58:31.893 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f897c89b700 1 Processor -- start 2023-10-18T17:58:31.893 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f897c89b700 1 -- start start 2023-10-18T17:58:31.894 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f897c89b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780c2460 0x7f8978159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.894 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f897c89b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8978151870 0x7f8978150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f897c89b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89781578f0 0x7f8978150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f897c89b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897815c880 con 0x7f89781578f0 2023-10-18T17:58:31.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f897c89b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89780b7540 con 0x7f89780c2460 2023-10-18T17:58:31.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f897c89b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8978168ca0 con 0x7f8978151870 2023-10-18T17:58:31.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8976d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89781578f0 0x7f8978150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8975d9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8978151870 0x7f8978150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.897 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f897659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780c2460 0x7f8978159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.897 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8976d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89781578f0 0x7f8978150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36386/0 (socket says 172.21.15.72:36386) 2023-10-18T17:58:31.897 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8975d9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8978151870 0x7f8978150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59244/0 (socket says 172.21.15.72:59244) 2023-10-18T17:58:31.897 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8976d9d700 1 -- 172.21.15.72:0/662177475 learned_addr learned my addr 172.21.15.72:0/662177475 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:31.898 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8976d9d700 1 -- 172.21.15.72:0/662177475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8978151870 msgr2=0x7f8978150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.898 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8976d9d700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8978151870 0x7f8978150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.898 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8976d9d700 1 -- 172.21.15.72:0/662177475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780c2460 msgr2=0x7f8978159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.898 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8976d9d700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89780c2460 0x7f8978159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.898 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8976d9d700 1 -- 172.21.15.72:0/662177475 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c009370 con 0x7f89781578f0 2023-10-18T17:58:31.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.890+0000 7f8976d9d700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89781578f0 0x7f8978150640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8970009e90 tx=0x7f897000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.894+0000 7f89667fc700 1 -- 172.21.15.72:0/662177475 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897000caa0 con 0x7f89781578f0 2023-10-18T17:58:31.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.894+0000 7f89667fc700 1 -- 172.21.15.72:0/662177475 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897000f040 con 0x7f89781578f0 2023-10-18T17:58:31.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.894+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89780b7280 con 0x7f89781578f0 2023-10-18T17:58:31.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.894+0000 7f89667fc700 1 -- 172.21.15.72:0/662177475 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897000ce10 con 0x7f89781578f0 2023-10-18T17:58:31.900 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.894+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8978168650 con 0x7f89781578f0 2023-10-18T17:58:31.900 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.894+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f897806c190 con 0x7f89781578f0 2023-10-18T17:58:31.900 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.894+0000 7f89667fc700 1 -- 172.21.15.72:0/662177475 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8970013610 con 0x7f89781578f0 2023-10-18T17:58:31.901 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.898+0000 7f89667fc700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f895805a540 0x7f895805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.901 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.898+0000 7f89667fc700 1 -- 172.21.15.72:0/662177475 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8970074ff0 con 0x7f89781578f0 2023-10-18T17:58:31.902 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.898+0000 7f897659c700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f895805a540 0x7f895805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.903 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.898+0000 7f89667fc700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.72:6811/33228,v1:172.21.15.72:6814/33228] conn(0x7f89580638e0 0x7f8958065d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:31.903 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.898+0000 7f89667fc700 1 -- 172.21.15.72:0/662177475 --> [v2:172.21.15.72:6811/33228,v1:172.21.15.72:6814/33228] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89580663c0 con 0x7f89580638e0 2023-10-18T17:58:31.903 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.898+0000 7f89667fc700 1 -- 172.21.15.72:0/662177475 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8970077590 con 0x7f89781578f0 2023-10-18T17:58:31.903 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.898+0000 7f8975d9b700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.72:6811/33228,v1:172.21.15.72:6814/33228] conn(0x7f89580638e0 0x7f8958065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:31.903 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.898+0000 7f897659c700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f895805a540 0x7f895805ca00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8968003860 tx=0x7f8968007d40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.898+0000 7f8975d9b700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.72:6811/33228,v1:172.21.15.72:6814/33228] conn(0x7f89580638e0 0x7f8958065d40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:31.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.902+0000 7f89667fc700 1 -- 172.21.15.72:0/662177475 <== osd.3 v2:172.21.15.72:6811/33228 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f89580663c0 con 0x7f89580638e0 2023-10-18T17:58:31.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 --> [v2:172.21.15.72:6811/33228,v1:172.21.15.72:6814/33228] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8978168de0 con 0x7f89580638e0 2023-10-18T17:58:31.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f89667fc700 1 -- 172.21.15.72:0/662177475 <== osd.3 v2:172.21.15.72:6811/33228 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8978168de0 con 0x7f89580638e0 2023-10-18T17:58:31.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 >> [v2:172.21.15.72:6811/33228,v1:172.21.15.72:6814/33228] conn(0x7f89580638e0 msgr2=0x7f8958065d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.72:6811/33228,v1:172.21.15.72:6814/33228] conn(0x7f89580638e0 0x7f8958065d40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:31.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f895805a540 msgr2=0x7f895805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f895805a540 0x7f895805ca00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8968003860 tx=0x7f8968007d40).stop 2023-10-18T17:58:31.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89781578f0 msgr2=0x7f8978150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:31.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 --2- 172.21.15.72:0/662177475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f89781578f0 0x7f8978150640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8970009e90 tx=0x7f897000b040).stop 2023-10-18T17:58:31.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f8976d9d700 1 -- 172.21.15.72:0/662177475 reap_dead start 2023-10-18T17:58:31.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 shutdown_connections 2023-10-18T17:58:31.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 >> 172.21.15.72:0/662177475 conn(0x7f897805dc20 msgr2=0x7f89780bf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:31.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 shutdown_connections 2023-10-18T17:58:31.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:31.918+0000 7f897c89b700 1 -- 172.21.15.72:0/662177475 wait complete. 2023-10-18T17:58:31.945 INFO:teuthology.orchestra.run.smithi072.stdout:60129542151 2023-10-18T17:58:31.946 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-18T17:58:32.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.106+0000 7f52fd85b700 1 Processor -- start 2023-10-18T17:58:32.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.106+0000 7f52fd85b700 1 -- start start 2023-10-18T17:58:32.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.106+0000 7f52fd85b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f52f80b2630 0x7f52f8150980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.106+0000 7f52fd85b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f8150e70 con 0x7f52f81534d0 2023-10-18T17:58:32.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52fd85b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f8153a10 con 0x7f52f80c2460 2023-10-18T17:58:32.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52fd85b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f81517b0 con 0x7f52f80b2630 2023-10-18T17:58:32.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f77fe700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f52f81534d0 0x7f52f80bb590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51116/0 (socket says 172.21.15.72:51116) 2023-10-18T17:58:32.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f77fe700 1 -- 172.21.15.72:0/2685639158 learned_addr learned my addr 172.21.15.72:0/2685639158 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:32.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f67fc700 1 --2- 172.21.15.72:0/2685639158 >> v2:172.21.15.72:3300/0 conn(0x7f52f80b2630 0x7f52f8150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f5ffb700 1 -- 172.21.15.72:0/2685639158 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138311222 0 0) 0x7f52f8150e70 con 0x7f52f81534d0 2023-10-18T17:58:32.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f5ffb700 1 -- 172.21.15.72:0/2685639158 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e00033f0 con 0x7f52f81534d0 2023-10-18T17:58:32.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f5ffb700 1 -- 172.21.15.72:0/2685639158 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115101731 0 0) 0x7f52f8153a10 con 0x7f52f80c2460 2023-10-18T17:58:32.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f5ffb700 1 -- 172.21.15.72:0/2685639158 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f8150e70 con 0x7f52f80c2460 2023-10-18T17:58:32.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f67fc700 1 -- 172.21.15.72:0/2685639158 >> v1:172.21.15.179:6789/0 conn(0x7f52f80c2460 legacy=0x7f52f80b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f5ffb700 1 -- 172.21.15.72:0/2685639158 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524491502 0 0) 0x7f52e00033f0 con 0x7f52f81534d0 2023-10-18T17:58:32.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f67fc700 1 -- 172.21.15.72:0/2685639158 >> v1:172.21.15.72:6789/0 conn(0x7f52f81534d0 legacy=0x7f52f80bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f67fc700 1 -- 172.21.15.72:0/2685639158 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f8153cf0 con 0x7f52f80b2630 2023-10-18T17:58:32.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f67fc700 1 --2- 172.21.15.72:0/2685639158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80b2630 0x7f52f8150980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f52ec009760 tx=0x7f52ec009060).ready entity=mon.0 client_cookie=8234fd30c0e3529f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f5ffb700 1 -- 172.21.15.72:0/2685639158 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0001810 con 0x7f52f80b2630 2023-10-18T17:58:32.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f5ffb700 1 -- 172.21.15.72:0/2685639158 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec011400 con 0x7f52f80b2630 2023-10-18T17:58:32.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52f5ffb700 1 -- 172.21.15.72:0/2685639158 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec011780 con 0x7f52f80b2630 2023-10-18T17:58:32.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52fd85b700 1 -- 172.21.15.72:0/2685639158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80b2630 msgr2=0x7f52f8150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52fd85b700 1 --2- 172.21.15.72:0/2685639158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80b2630 0x7f52f8150980 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f52ec009760 tx=0x7f52ec009060).stop 2023-10-18T17:58:32.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52fd85b700 1 -- 172.21.15.72:0/2685639158 shutdown_connections 2023-10-18T17:58:32.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52fd85b700 1 --2- 172.21.15.72:0/2685639158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80b2630 0x7f52f8150980 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52fd85b700 1 -- 172.21.15.72:0/2685639158 >> 172.21.15.72:0/2685639158 conn(0x7f52f805dc20 msgr2=0x7f52f805e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:32.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52fd85b700 1 -- 172.21.15.72:0/2685639158 shutdown_connections 2023-10-18T17:58:32.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.110+0000 7f52fd85b700 1 -- 172.21.15.72:0/2685639158 wait complete. 2023-10-18T17:58:32.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52fd85b700 1 Processor -- start 2023-10-18T17:58:32.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52fd85b700 1 -- start start 2023-10-18T17:58:32.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52fd85b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f80b2630 0x7f52f80b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52fd85b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80c2460 0x7f52f80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52fd85b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f81534d0 0x7f52f8145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52fd85b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b0b50 con 0x7f52f80c2460 2023-10-18T17:58:32.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52fd85b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b1ef0 con 0x7f52f80b2630 2023-10-18T17:58:32.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52fd85b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f81544f0 con 0x7f52f81534d0 2023-10-18T17:58:32.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52f67fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80c2460 0x7f52f80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52f67fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80c2460 0x7f52f80b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36392/0 (socket says 172.21.15.72:36392) 2023-10-18T17:58:32.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52f67fc700 1 -- 172.21.15.72:0/4275533632 learned_addr learned my addr 172.21.15.72:0/4275533632 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:32.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.114+0000 7f52f77fe700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f81534d0 0x7f52f8145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52f6ffd700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f80b2630 0x7f52f80b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52f67fc700 1 -- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f81534d0 msgr2=0x7f52f8145920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52f67fc700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f52f81534d0 0x7f52f8145920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52f67fc700 1 -- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f80b2630 msgr2=0x7f52f80b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52f67fc700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52f80b2630 0x7f52f80b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52f67fc700 1 -- 172.21.15.72:0/4275533632 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec0092a0 con 0x7f52f80c2460 2023-10-18T17:58:32.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52f67fc700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80c2460 0x7f52f80b43b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f52ec010000 tx=0x7f52ec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52deffd700 1 -- 172.21.15.72:0/4275533632 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec010740 con 0x7f52f80c2460 2023-10-18T17:58:32.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52deffd700 1 -- 172.21.15.72:0/4275533632 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec0108a0 con 0x7f52f80c2460 2023-10-18T17:58:32.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52deffd700 1 -- 172.21.15.72:0/4275533632 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec010c40 con 0x7f52f80c2460 2023-10-18T17:58:32.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f80af4b0 con 0x7f52f80c2460 2023-10-18T17:58:32.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f80b2380 con 0x7f52f80c2460 2023-10-18T17:58:32.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.118+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52f806c190 con 0x7f52f80c2460 2023-10-18T17:58:32.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.122+0000 7f52deffd700 1 -- 172.21.15.72:0/4275533632 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f52ec027460 con 0x7f52f80c2460 2023-10-18T17:58:32.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.122+0000 7f52deffd700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f52e405a540 0x7f52e405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.122+0000 7f52deffd700 1 -- 172.21.15.72:0/4275533632 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f52ec075e40 con 0x7f52f80c2460 2023-10-18T17:58:32.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.122+0000 7f52f6ffd700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f52e405a540 0x7f52e405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.122+0000 7f52deffd700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6808/34034,v1:172.21.15.179:6809/34034] conn(0x7f52e40638e0 0x7f52e4065d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.122+0000 7f52deffd700 1 -- 172.21.15.72:0/4275533632 --> [v2:172.21.15.179:6808/34034,v1:172.21.15.179:6809/34034] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52e40663c0 con 0x7f52e40638e0 2023-10-18T17:58:32.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.122+0000 7f52deffd700 1 -- 172.21.15.72:0/4275533632 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52ec0783a0 con 0x7f52f80c2460 2023-10-18T17:58:32.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.122+0000 7f52f77fe700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6808/34034,v1:172.21.15.179:6809/34034] conn(0x7f52e40638e0 0x7f52e4065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.122+0000 7f52f6ffd700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f52e405a540 0x7f52e405ca00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f52e8003930 tx=0x7f52e8007bd0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.131 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.126+0000 7f52f77fe700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6808/34034,v1:172.21.15.179:6809/34034] conn(0x7f52e40638e0 0x7f52e4065d40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.126+0000 7f52deffd700 1 -- 172.21.15.72:0/4275533632 <== osd.4 v2:172.21.15.179:6808/34034 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f52e40663c0 con 0x7f52e40638e0 2023-10-18T17:58:32.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 --> [v2:172.21.15.179:6808/34034,v1:172.21.15.179:6809/34034] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52f805e740 con 0x7f52e40638e0 2023-10-18T17:58:32.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52deffd700 1 -- 172.21.15.72:0/4275533632 <== osd.4 v2:172.21.15.179:6808/34034 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f52f805e740 con 0x7f52e40638e0 2023-10-18T17:58:32.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6808/34034,v1:172.21.15.179:6809/34034] conn(0x7f52e40638e0 msgr2=0x7f52e4065d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6808/34034,v1:172.21.15.179:6809/34034] conn(0x7f52e40638e0 0x7f52e4065d40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f52e405a540 msgr2=0x7f52e405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f52e405a540 0x7f52e405ca00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f52e8003930 tx=0x7f52e8007bd0).stop 2023-10-18T17:58:32.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80c2460 msgr2=0x7f52f80b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 --2- 172.21.15.72:0/4275533632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f52f80c2460 0x7f52f80b43b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f52ec010000 tx=0x7f52ec016000).stop 2023-10-18T17:58:32.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52f77fe700 1 -- 172.21.15.72:0/4275533632 reap_dead start 2023-10-18T17:58:32.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 shutdown_connections 2023-10-18T17:58:32.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 >> 172.21.15.72:0/4275533632 conn(0x7f52f805dc20 msgr2=0x7f52f80bc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:32.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 shutdown_connections 2023-10-18T17:58:32.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.146+0000 7f52fd85b700 1 -- 172.21.15.72:0/4275533632 wait complete. 2023-10-18T17:58:32.172 INFO:teuthology.orchestra.run.smithi072.stdout:55834574854 2023-10-18T17:58:32.173 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-18T17:58:32.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.334+0000 7fb846039700 1 Processor -- start 2023-10-18T17:58:32.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.334+0000 7fb846039700 1 -- start start 2023-10-18T17:58:32.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb846039700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8400bf720 0x7fb8400bcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb846039700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb8401539b0 0x7fb840153e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb846039700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb840153520 con 0x7fb8400bf340 2023-10-18T17:58:32.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb846039700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb840154330 con 0x7fb8401539b0 2023-10-18T17:58:32.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb846039700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb840068150 con 0x7fb8400bf720 2023-10-18T17:58:32.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb845037700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fb8400bf340 0x7fb840066240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51118/0 (socket says 172.21.15.72:51118) 2023-10-18T17:58:32.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb845037700 1 -- 172.21.15.72:0/2147130829 learned_addr learned my addr 172.21.15.72:0/2147130829 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:32.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb844836700 1 --2- 172.21.15.72:0/2147130829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8400bf720 0x7fb8400bcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb845838700 1 --2- 172.21.15.72:0/2147130829 >> v2:172.21.15.179:3300/0 conn(0x7fb8401539b0 0x7fb840153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb837fff700 1 -- 172.21.15.72:0/2147130829 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741852518 0 0) 0x7fb840153520 con 0x7fb8400bf340 2023-10-18T17:58:32.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb837fff700 1 -- 172.21.15.72:0/2147130829 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb82c0033f0 con 0x7fb8400bf340 2023-10-18T17:58:32.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb844836700 1 -- 172.21.15.72:0/2147130829 >> v2:172.21.15.179:3300/0 conn(0x7fb8401539b0 msgr2=0x7fb840153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:32.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb844836700 1 --2- 172.21.15.72:0/2147130829 >> v2:172.21.15.179:3300/0 conn(0x7fb8401539b0 0x7fb840153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb844836700 1 -- 172.21.15.72:0/2147130829 >> v1:172.21.15.72:6789/0 conn(0x7fb8400bf340 legacy=0x7fb840066240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb844836700 1 -- 172.21.15.72:0/2147130829 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840068430 con 0x7fb8400bf720 2023-10-18T17:58:32.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb845838700 1 --2- 172.21.15.72:0/2147130829 >> v2:172.21.15.179:3300/0 conn(0x7fb8401539b0 0x7fb840153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:32.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb844836700 1 --2- 172.21.15.72:0/2147130829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8400bf720 0x7fb8400bcea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb830009760 tx=0x7fb830009060).ready entity=mon.2 client_cookie=e6bb46c3875c3dd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb837fff700 1 -- 172.21.15.72:0/2147130829 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000f510 con 0x7fb8400bf720 2023-10-18T17:58:32.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb837fff700 1 -- 172.21.15.72:0/2147130829 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83000f670 con 0x7fb8400bf720 2023-10-18T17:58:32.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.338+0000 7fb837fff700 1 -- 172.21.15.72:0/2147130829 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000f9f0 con 0x7fb8400bf720 2023-10-18T17:58:32.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 -- 172.21.15.72:0/2147130829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8400bf720 msgr2=0x7fb8400bcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 --2- 172.21.15.72:0/2147130829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8400bf720 0x7fb8400bcea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb830009760 tx=0x7fb830009060).stop 2023-10-18T17:58:32.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 -- 172.21.15.72:0/2147130829 shutdown_connections 2023-10-18T17:58:32.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 --2- 172.21.15.72:0/2147130829 >> v2:172.21.15.179:3300/0 conn(0x7fb8401539b0 0x7fb840153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 --2- 172.21.15.72:0/2147130829 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8400bf720 0x7fb8400bcea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 -- 172.21.15.72:0/2147130829 >> 172.21.15.72:0/2147130829 conn(0x7fb84005ddc0 msgr2=0x7fb84005e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:32.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 -- 172.21.15.72:0/2147130829 shutdown_connections 2023-10-18T17:58:32.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 -- 172.21.15.72:0/2147130829 wait complete. 2023-10-18T17:58:32.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 Processor -- start 2023-10-18T17:58:32.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.342+0000 7fb846039700 1 -- start start 2023-10-18T17:58:32.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb846039700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8400bf340 0x7fb8400c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb846039700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8400bf720 0x7fb840152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb846039700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8401539b0 0x7fb840147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb846039700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84015fe80 con 0x7fb8400bf720 2023-10-18T17:58:32.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb846039700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb840146bf0 con 0x7fb8400bf340 2023-10-18T17:58:32.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb846039700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb840160070 con 0x7fb8401539b0 2023-10-18T17:58:32.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb844836700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8400bf720 0x7fb840152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb844836700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8400bf720 0x7fb840152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36402/0 (socket says 172.21.15.72:36402) 2023-10-18T17:58:32.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb844836700 1 -- 172.21.15.72:0/2760333759 learned_addr learned my addr 172.21.15.72:0/2760333759 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:32.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb845838700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8401539b0 0x7fb840147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb845037700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8400bf340 0x7fb8400c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb844836700 1 -- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8401539b0 msgr2=0x7fb840147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb844836700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8401539b0 0x7fb840147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb844836700 1 -- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8400bf340 msgr2=0x7fb8400c2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb844836700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8400bf340 0x7fb8400c2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb844836700 1 -- 172.21.15.72:0/2760333759 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb838009a20 con 0x7fb8400bf720 2023-10-18T17:58:32.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb844836700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8400bf720 0x7fb840152390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb830009d50 tx=0x7fb830017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb8357fa700 1 -- 172.21.15.72:0/2760333759 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000f7e0 con 0x7fb8400bf720 2023-10-18T17:58:32.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb8357fa700 1 -- 172.21.15.72:0/2760333759 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83000f940 con 0x7fb8400bf720 2023-10-18T17:58:32.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8300092a0 con 0x7fb8400bf720 2023-10-18T17:58:32.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb8357fa700 1 -- 172.21.15.72:0/2760333759 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83001b660 con 0x7fb8400bf720 2023-10-18T17:58:32.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.346+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb840160630 con 0x7fb8400bf720 2023-10-18T17:58:32.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb840046f00 con 0x7fb8400bf720 2023-10-18T17:58:32.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb8357fa700 1 -- 172.21.15.72:0/2760333759 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb83001ba50 con 0x7fb8400bf720 2023-10-18T17:58:32.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb8357fa700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb82405a510 0x7fb82405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb8357fa700 1 -- 172.21.15.72:0/2760333759 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb8300760f0 con 0x7fb8400bf720 2023-10-18T17:58:32.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb845037700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb82405a510 0x7fb82405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb8357fa700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6800/34044,v1:172.21.15.179:6801/34044] conn(0x7fb8240638b0 0x7fb824065d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb8357fa700 1 -- 172.21.15.72:0/2760333759 --> [v2:172.21.15.179:6800/34044,v1:172.21.15.179:6801/34044] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb824066390 con 0x7fb8240638b0 2023-10-18T17:58:32.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb8357fa700 1 -- 172.21.15.72:0/2760333759 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb830078650 con 0x7fb8400bf720 2023-10-18T17:58:32.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb845838700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6800/34044,v1:172.21.15.179:6801/34044] conn(0x7fb8240638b0 0x7fb824065d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.350+0000 7fb845037700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb82405a510 0x7fb82405c9d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb83c005750 tx=0x7fb83c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.354+0000 7fb845838700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6800/34044,v1:172.21.15.179:6801/34044] conn(0x7fb8240638b0 0x7fb824065d10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.354+0000 7fb8357fa700 1 -- 172.21.15.72:0/2760333759 <== osd.5 v2:172.21.15.179:6800/34044 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fb824066390 con 0x7fb8240638b0 2023-10-18T17:58:32.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.374+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 --> [v2:172.21.15.179:6800/34044,v1:172.21.15.179:6801/34044] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb840146e20 con 0x7fb8240638b0 2023-10-18T17:58:32.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.374+0000 7fb8357fa700 1 -- 172.21.15.72:0/2760333759 <== osd.5 v2:172.21.15.179:6800/34044 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb840146e20 con 0x7fb8240638b0 2023-10-18T17:58:32.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.374+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6800/34044,v1:172.21.15.179:6801/34044] conn(0x7fb8240638b0 msgr2=0x7fb824065d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.374+0000 7fb846039700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6800/34044,v1:172.21.15.179:6801/34044] conn(0x7fb8240638b0 0x7fb824065d10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.374+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb82405a510 msgr2=0x7fb82405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.374+0000 7fb846039700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb82405a510 0x7fb82405c9d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb83c005750 tx=0x7fb83c009040).stop 2023-10-18T17:58:32.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.374+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8400bf720 msgr2=0x7fb840152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.374+0000 7fb846039700 1 --2- 172.21.15.72:0/2760333759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8400bf720 0x7fb840152390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb830009d50 tx=0x7fb830017000).stop 2023-10-18T17:58:32.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.374+0000 7fb845838700 1 -- 172.21.15.72:0/2760333759 reap_dead start 2023-10-18T17:58:32.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.378+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 shutdown_connections 2023-10-18T17:58:32.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.378+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 >> 172.21.15.72:0/2760333759 conn(0x7fb84005ddc0 msgr2=0x7fb8400bc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:32.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.378+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 shutdown_connections 2023-10-18T17:58:32.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.378+0000 7fb846039700 1 -- 172.21.15.72:0/2760333759 wait complete. 2023-10-18T17:58:32.402 INFO:teuthology.orchestra.run.smithi072.stdout:55834574854 2023-10-18T17:58:32.402 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-18T17:58:32.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.558+0000 7f354908f700 1 Processor -- start 2023-10-18T17:58:32.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f354908f700 1 -- start start 2023-10-18T17:58:32.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f354908f700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3544151820 0x7f3544151c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f354908f700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354414eae0 con 0x7f35440bf340 2023-10-18T17:58:32.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f354908f700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35441520f0 con 0x7f35440bf720 2023-10-18T17:58:32.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f354908f700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354414dc30 con 0x7f3544151820 2023-10-18T17:58:32.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f3543fff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f35440bf340 0x7f354414e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51126/0 (socket says 172.21.15.72:51126) 2023-10-18T17:58:32.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f3543fff700 1 -- 172.21.15.72:0/2076295157 learned_addr learned my addr 172.21.15.72:0/2076295157 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:32.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f354888e700 1 --2- 172.21.15.72:0/2076295157 >> v2:172.21.15.72:3300/0 conn(0x7f3544151820 0x7f3544151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f35437fe700 1 -- 172.21.15.72:0/2076295157 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866981515 0 0) 0x7f35441520f0 con 0x7f35440bf720 2023-10-18T17:58:32.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f35437fe700 1 -- 172.21.15.72:0/2076295157 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35300033f0 con 0x7f35440bf720 2023-10-18T17:58:32.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f35437fe700 1 -- 172.21.15.72:0/2076295157 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034161470 0 0) 0x7f354414eae0 con 0x7f35440bf340 2023-10-18T17:58:32.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f354888e700 1 -- 172.21.15.72:0/2076295157 >> v1:172.21.15.179:6789/0 conn(0x7f35440bf720 legacy=0x7f3544066180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f354888e700 1 -- 172.21.15.72:0/2076295157 >> v1:172.21.15.72:6789/0 conn(0x7f35440bf340 legacy=0x7f354414e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.562+0000 7f354888e700 1 -- 172.21.15.72:0/2076295157 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35440bbb60 con 0x7f3544151820 2023-10-18T17:58:32.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f354888e700 1 --2- 172.21.15.72:0/2076295157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3544151820 0x7f3544151c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f353c009e60 tx=0x7f353c009780).ready entity=mon.0 client_cookie=7da852b5f4fcd3bf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f35437fe700 1 -- 172.21.15.72:0/2076295157 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c00e070 con 0x7f3544151820 2023-10-18T17:58:32.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f35437fe700 1 -- 172.21.15.72:0/2076295157 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c00bbf0 con 0x7f3544151820 2023-10-18T17:58:32.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f35437fe700 1 -- 172.21.15.72:0/2076295157 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c012410 con 0x7f3544151820 2023-10-18T17:58:32.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f354908f700 1 -- 172.21.15.72:0/2076295157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3544151820 msgr2=0x7f3544151c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f354908f700 1 --2- 172.21.15.72:0/2076295157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3544151820 0x7f3544151c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f353c009e60 tx=0x7f353c009780).stop 2023-10-18T17:58:32.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f354908f700 1 -- 172.21.15.72:0/2076295157 shutdown_connections 2023-10-18T17:58:32.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f354908f700 1 --2- 172.21.15.72:0/2076295157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3544151820 0x7f3544151c00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f354908f700 1 -- 172.21.15.72:0/2076295157 >> 172.21.15.72:0/2076295157 conn(0x7f354405ddc0 msgr2=0x7f354405e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:32.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f354908f700 1 -- 172.21.15.72:0/2076295157 shutdown_connections 2023-10-18T17:58:32.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.566+0000 7f354908f700 1 -- 172.21.15.72:0/2076295157 wait complete. 2023-10-18T17:58:32.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f354908f700 1 Processor -- start 2023-10-18T17:58:32.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f354908f700 1 -- start start 2023-10-18T17:58:32.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f354908f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35440bf340 0x7f3544144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f354908f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35440bf720 0x7f35441639c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f354908f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3544151820 0x7f3544163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f354908f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3544150350 con 0x7f35440bf720 2023-10-18T17:58:32.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f354908f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354414aa10 con 0x7f35440bf340 2023-10-18T17:58:32.576 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f354908f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35440c01e0 con 0x7f3544151820 2023-10-18T17:58:32.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3543fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35440bf340 0x7f3544144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3543fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35440bf340 0x7f3544144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:56264/0 (socket says 172.21.15.72:56264) 2023-10-18T17:58:32.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f354888e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3544151820 0x7f3544163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3543fff700 1 -- 172.21.15.72:0/3217999072 learned_addr learned my addr 172.21.15.72:0/3217999072 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:32.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f353bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35440bf720 0x7f35441639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3543fff700 1 -- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3544151820 msgr2=0x7f3544163eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3543fff700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3544151820 0x7f3544163eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3543fff700 1 -- 172.21.15.72:0/3217999072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35440bf720 msgr2=0x7f35441639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3543fff700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35440bf720 0x7f35441639c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3543fff700 1 -- 172.21.15.72:0/3217999072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c009a20 con 0x7f35440bf340 2023-10-18T17:58:32.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3543fff700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35440bf340 0x7f3544144af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f353400c4d0 tx=0x7f3534007c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.570+0000 7f3540ff9700 1 -- 172.21.15.72:0/3217999072 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353400ce40 con 0x7f35440bf340 2023-10-18T17:58:32.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.574+0000 7f3540ff9700 1 -- 172.21.15.72:0/3217999072 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35340072e0 con 0x7f35440bf340 2023-10-18T17:58:32.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.574+0000 7f3540ff9700 1 -- 172.21.15.72:0/3217999072 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534007680 con 0x7f35440bf340 2023-10-18T17:58:32.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.574+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35441643a0 con 0x7f35440bf340 2023-10-18T17:58:32.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.574+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3544147b50 con 0x7f35440bf340 2023-10-18T17:58:32.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.574+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3544163600 con 0x7f35440bf340 2023-10-18T17:58:32.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.574+0000 7f3540ff9700 1 -- 172.21.15.72:0/3217999072 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3534030a40 con 0x7f35440bf340 2023-10-18T17:58:32.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.578+0000 7f3540ff9700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f352405a510 0x7f352405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.578+0000 7f3540ff9700 1 -- 172.21.15.72:0/3217999072 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3534074090 con 0x7f35440bf340 2023-10-18T17:58:32.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.578+0000 7f353bfff700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f352405a510 0x7f352405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.578+0000 7f3540ff9700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6824/34036,v1:172.21.15.179:6825/34036] conn(0x7f35240638b0 0x7f3524065d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.578+0000 7f3540ff9700 1 -- 172.21.15.72:0/3217999072 --> [v2:172.21.15.179:6824/34036,v1:172.21.15.179:6825/34036] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3524066390 con 0x7f35240638b0 2023-10-18T17:58:32.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.578+0000 7f3540ff9700 1 -- 172.21.15.72:0/3217999072 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3534074380 con 0x7f35440bf340 2023-10-18T17:58:32.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.578+0000 7f354888e700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6824/34036,v1:172.21.15.179:6825/34036] conn(0x7f35240638b0 0x7f3524065d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.578+0000 7f353bfff700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f352405a510 0x7f352405c9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f35441520a0 tx=0x7f352c007bb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.578+0000 7f354888e700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6824/34036,v1:172.21.15.179:6825/34036] conn(0x7f35240638b0 0x7f3524065d10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.582+0000 7f3540ff9700 1 -- 172.21.15.72:0/3217999072 <== osd.6 v2:172.21.15.179:6824/34036 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f3524066390 con 0x7f35240638b0 2023-10-18T17:58:32.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.598+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 --> [v2:172.21.15.179:6824/34036,v1:172.21.15.179:6825/34036] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3544046f00 con 0x7f35240638b0 2023-10-18T17:58:32.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f3540ff9700 1 -- 172.21.15.72:0/3217999072 <== osd.6 v2:172.21.15.179:6824/34036 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3544046f00 con 0x7f35240638b0 2023-10-18T17:58:32.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6824/34036,v1:172.21.15.179:6825/34036] conn(0x7f35240638b0 msgr2=0x7f3524065d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6824/34036,v1:172.21.15.179:6825/34036] conn(0x7f35240638b0 0x7f3524065d10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f352405a510 msgr2=0x7f352405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f352405a510 0x7f352405c9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f35441520a0 tx=0x7f352c007bb0).stop 2023-10-18T17:58:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35440bf340 msgr2=0x7f3544144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 --2- 172.21.15.72:0/3217999072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35440bf340 0x7f3544144af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f353400c4d0 tx=0x7f3534007c50).stop 2023-10-18T17:58:32.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354888e700 1 -- 172.21.15.72:0/3217999072 reap_dead start 2023-10-18T17:58:32.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 shutdown_connections 2023-10-18T17:58:32.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 >> 172.21.15.72:0/3217999072 conn(0x7f354405ddc0 msgr2=0x7f35440654c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:32.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 shutdown_connections 2023-10-18T17:58:32.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.602+0000 7f354908f700 1 -- 172.21.15.72:0/3217999072 wait complete. 2023-10-18T17:58:32.623 INFO:teuthology.orchestra.run.smithi072.stdout:55834574854 2023-10-18T17:58:32.624 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-18T17:58:32.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.786+0000 7fdf44b07700 1 Processor -- start 2023-10-18T17:58:32.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.786+0000 7fdf44b07700 1 -- start start 2023-10-18T17:58:32.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bf340 0x7fdf400bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdf4013d890 0x7fdf400689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf40068ea0 con 0x7fdf400bcaf0 2023-10-18T17:58:32.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf40154220 con 0x7fdf4013d890 2023-10-18T17:58:32.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40157970 con 0x7fdf400bf340 2023-10-18T17:58:32.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3f7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bf340 0x7fdf400bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdf4013d890 0x7fdf400689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3f7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bf340 0x7fdf400bf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59284/0 (socket says 172.21.15.72:59284) 2023-10-18T17:58:32.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3f7fe700 1 -- 172.21.15.72:0/1859911306 learned_addr learned my addr 172.21.15.72:0/1859911306 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:32.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3effd700 1 --1- 172.21.15.72:0/1859911306 >> v1:172.21.15.72:6789/0 conn(0x7fdf400bcaf0 0x7fdf4013d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51142/0 (socket says 172.21.15.72:51142) 2023-10-18T17:58:32.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3f7fe700 1 -- 172.21.15.72:0/1859911306 >> v2:172.21.15.179:3300/0 conn(0x7fdf4013d890 msgr2=0x7fdf400689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:32.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3f7fe700 1 --2- 172.21.15.72:0/1859911306 >> v2:172.21.15.179:3300/0 conn(0x7fdf4013d890 0x7fdf400689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3f7fe700 1 -- 172.21.15.72:0/1859911306 >> v1:172.21.15.72:6789/0 conn(0x7fdf400bcaf0 legacy=0x7fdf4013d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3f7fe700 1 -- 172.21.15.72:0/1859911306 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf40157bf0 con 0x7fdf400bf340 2023-10-18T17:58:32.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3f7fe700 1 --2- 172.21.15.72:0/1859911306 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bf340 0x7fdf400bf720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fdf30006fd0 tx=0x7fdf300090f0).ready entity=mon.2 client_cookie=558980fb96c15989 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3e7fc700 1 -- 172.21.15.72:0/1859911306 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000b610 con 0x7fdf400bf340 2023-10-18T17:58:32.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3e7fc700 1 -- 172.21.15.72:0/1859911306 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3000b770 con 0x7fdf400bf340 2023-10-18T17:58:32.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf3e7fc700 1 -- 172.21.15.72:0/1859911306 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000baf0 con 0x7fdf400bf340 2023-10-18T17:58:32.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 -- 172.21.15.72:0/1859911306 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bf340 msgr2=0x7fdf400bf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 --2- 172.21.15.72:0/1859911306 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bf340 0x7fdf400bf720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fdf30006fd0 tx=0x7fdf300090f0).stop 2023-10-18T17:58:32.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 -- 172.21.15.72:0/1859911306 shutdown_connections 2023-10-18T17:58:32.800 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 --2- 172.21.15.72:0/1859911306 >> v2:172.21.15.179:3300/0 conn(0x7fdf4013d890 0x7fdf400689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.800 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 --2- 172.21.15.72:0/1859911306 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bf340 0x7fdf400bf720 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.800 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 -- 172.21.15.72:0/1859911306 >> 172.21.15.72:0/1859911306 conn(0x7fdf4005ddc0 msgr2=0x7fdf4005e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:32.800 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 -- 172.21.15.72:0/1859911306 shutdown_connections 2023-10-18T17:58:32.800 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.790+0000 7fdf44b07700 1 -- 172.21.15.72:0/1859911306 wait complete. 2023-10-18T17:58:32.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.794+0000 7fdf44b07700 1 Processor -- start 2023-10-18T17:58:32.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.794+0000 7fdf44b07700 1 -- start start 2023-10-18T17:58:32.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf44b07700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bcaf0 0x7fdf40164370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf44b07700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf400bf340 0x7fdf40164860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf44b07700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4013d890 0x7fdf40150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf44b07700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40157600 con 0x7fdf400bf340 2023-10-18T17:58:32.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf44b07700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40152280 con 0x7fdf4013d890 2023-10-18T17:58:32.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf44b07700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400b9e40 con 0x7fdf400bcaf0 2023-10-18T17:58:32.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3f7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bcaf0 0x7fdf40164370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4013d890 0x7fdf40150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3f7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bcaf0 0x7fdf40164370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59290/0 (socket says 172.21.15.72:59290) 2023-10-18T17:58:32.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf400bf340 0x7fdf40164860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3f7fe700 1 -- 172.21.15.72:0/950518835 learned_addr learned my addr 172.21.15.72:0/950518835 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:32.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3f7fe700 1 -- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4013d890 msgr2=0x7fdf40150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3f7fe700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4013d890 0x7fdf40150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3f7fe700 1 -- 172.21.15.72:0/950518835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf400bf340 msgr2=0x7fdf40164860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3f7fe700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf400bf340 0x7fdf40164860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3f7fe700 1 -- 172.21.15.72:0/950518835 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30009370 con 0x7fdf400bcaf0 2023-10-18T17:58:32.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3effd700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf400bf340 0x7fdf40164860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:32.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3ffff700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4013d890 0x7fdf40150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:32.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf3f7fe700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bcaf0 0x7fdf40164370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fdf30009fd0 tx=0x7fdf30017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf277fe700 1 -- 172.21.15.72:0/950518835 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000beb0 con 0x7fdf400bcaf0 2023-10-18T17:58:32.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf277fe700 1 -- 172.21.15.72:0/950518835 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3000b8e0 con 0x7fdf400bcaf0 2023-10-18T17:58:32.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.798+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf40062c70 con 0x7fdf400bcaf0 2023-10-18T17:58:32.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.802+0000 7fdf277fe700 1 -- 172.21.15.72:0/950518835 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3001c670 con 0x7fdf400bcaf0 2023-10-18T17:58:32.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.802+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf400ba2d0 con 0x7fdf400bcaf0 2023-10-18T17:58:32.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.802+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf400b9690 con 0x7fdf400bcaf0 2023-10-18T17:58:32.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.802+0000 7fdf277fe700 1 -- 172.21.15.72:0/950518835 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdf3001ca20 con 0x7fdf400bcaf0 2023-10-18T17:58:32.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.806+0000 7fdf277fe700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdf2c05a540 0x7fdf2c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.806+0000 7fdf277fe700 1 -- 172.21.15.72:0/950518835 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdf30076e80 con 0x7fdf400bcaf0 2023-10-18T17:58:32.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.806+0000 7fdf3effd700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdf2c05a540 0x7fdf2c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.806+0000 7fdf277fe700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6810/34035,v1:172.21.15.179:6812/34035] conn(0x7fdf2c0638e0 0x7fdf2c065d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:32.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.806+0000 7fdf277fe700 1 -- 172.21.15.72:0/950518835 --> [v2:172.21.15.179:6810/34035,v1:172.21.15.179:6812/34035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf2c0663c0 con 0x7fdf2c0638e0 2023-10-18T17:58:32.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.806+0000 7fdf277fe700 1 -- 172.21.15.72:0/950518835 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf30077170 con 0x7fdf400bcaf0 2023-10-18T17:58:32.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.806+0000 7fdf3ffff700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6810/34035,v1:172.21.15.179:6812/34035] conn(0x7fdf2c0638e0 0x7fdf2c065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:32.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.806+0000 7fdf3effd700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdf2c05a540 0x7fdf2c05ca00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fdf401541b0 tx=0x7fdf34007d30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.806+0000 7fdf3ffff700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6810/34035,v1:172.21.15.179:6812/34035] conn(0x7fdf2c0638e0 0x7fdf2c065d40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:32.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.810+0000 7fdf277fe700 1 -- 172.21.15.72:0/950518835 <== osd.7 v2:172.21.15.179:6810/34035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fdf2c0663c0 con 0x7fdf2c0638e0 2023-10-18T17:58:32.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.826+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 --> [v2:172.21.15.179:6810/34035,v1:172.21.15.179:6812/34035] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf40046f00 con 0x7fdf2c0638e0 2023-10-18T17:58:32.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf277fe700 1 -- 172.21.15.72:0/950518835 <== osd.7 v2:172.21.15.179:6810/34035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdf40046f00 con 0x7fdf2c0638e0 2023-10-18T17:58:32.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6810/34035,v1:172.21.15.179:6812/34035] conn(0x7fdf2c0638e0 msgr2=0x7fdf2c065d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6810/34035,v1:172.21.15.179:6812/34035] conn(0x7fdf2c0638e0 0x7fdf2c065d40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:32.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdf2c05a540 msgr2=0x7fdf2c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdf2c05a540 0x7fdf2c05ca00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fdf401541b0 tx=0x7fdf34007d30).stop 2023-10-18T17:58:32.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bcaf0 msgr2=0x7fdf40164370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:32.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 --2- 172.21.15.72:0/950518835 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdf400bcaf0 0x7fdf40164370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fdf30009fd0 tx=0x7fdf30017000).stop 2023-10-18T17:58:32.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf3ffff700 1 -- 172.21.15.72:0/950518835 reap_dead start 2023-10-18T17:58:32.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 shutdown_connections 2023-10-18T17:58:32.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 >> 172.21.15.72:0/950518835 conn(0x7fdf4005ddc0 msgr2=0x7fdf400651e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:32.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 shutdown_connections 2023-10-18T17:58:32.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:32.830+0000 7fdf44b07700 1 -- 172.21.15.72:0/950518835 wait complete. 2023-10-18T17:58:32.855 INFO:teuthology.orchestra.run.smithi072.stdout:55834574853 2023-10-18T17:58:32.855 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T17:58:33.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.022+0000 7f7edd1b9700 1 Processor -- start 2023-10-18T17:58:33.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7edd1b9700 1 -- start start 2023-10-18T17:58:33.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7edd1b9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f7ed80bf720 0x7f7ed8066180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7edd1b9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f7ed8066670 0x7f7ed8151820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7edd1b9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed814eae0 con 0x7f7ed80bf340 2023-10-18T17:58:33.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7edd1b9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed8066a50 con 0x7f7ed80bf720 2023-10-18T17:58:33.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7edd1b9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed814dc30 con 0x7f7ed8066670 2023-10-18T17:58:33.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7ed7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f7ed80bf340 0x7f7ed814e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:33494/0 (socket says 172.21.15.72:33494) 2023-10-18T17:58:33.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7ed7fff700 1 -- 172.21.15.72:0/4121966308 learned_addr learned my addr 172.21.15.72:0/4121966308 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:33.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7ed77fe700 1 --2- 172.21.15.72:0/4121966308 >> v2:172.21.15.72:3300/0 conn(0x7f7ed80bf720 0x7f7ed8066180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.026+0000 7f7edc9b8700 1 --2- 172.21.15.72:0/4121966308 >> v2:172.21.15.179:3300/0 conn(0x7f7ed8066670 0x7f7ed8151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7ed77fe700 1 -- 172.21.15.72:0/4121966308 >> v2:172.21.15.179:3300/0 conn(0x7f7ed8066670 msgr2=0x7f7ed8151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:33.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7ed77fe700 1 --2- 172.21.15.72:0/4121966308 >> v2:172.21.15.179:3300/0 conn(0x7f7ed8066670 0x7f7ed8151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7ed6ffd700 1 -- 172.21.15.72:0/4121966308 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612725254 0 0) 0x7f7ed814eae0 con 0x7f7ed80bf340 2023-10-18T17:58:33.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7ed77fe700 1 -- 172.21.15.72:0/4121966308 >> v1:172.21.15.179:6789/0 conn(0x7f7ed80bf340 legacy=0x7f7ed814e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7ed77fe700 1 -- 172.21.15.72:0/4121966308 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed80bbb60 con 0x7f7ed80bf720 2023-10-18T17:58:33.035 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7ed77fe700 1 --2- 172.21.15.72:0/4121966308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf720 0x7f7ed8066180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f7ed8067da0 tx=0x7f7ec0009060).ready entity=mon.0 client_cookie=537b052d953723e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:33.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7ed6ffd700 1 -- 172.21.15.72:0/4121966308 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec000f510 con 0x7f7ed80bf720 2023-10-18T17:58:33.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7ed6ffd700 1 -- 172.21.15.72:0/4121966308 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec000f670 con 0x7f7ed80bf720 2023-10-18T17:58:33.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7ed6ffd700 1 -- 172.21.15.72:0/4121966308 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec000f9f0 con 0x7f7ed80bf720 2023-10-18T17:58:33.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7edd1b9700 1 -- 172.21.15.72:0/4121966308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf720 msgr2=0x7f7ed8066180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7edd1b9700 1 --2- 172.21.15.72:0/4121966308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf720 0x7f7ed8066180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f7ed8067da0 tx=0x7f7ec0009060).stop 2023-10-18T17:58:33.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7edd1b9700 1 -- 172.21.15.72:0/4121966308 shutdown_connections 2023-10-18T17:58:33.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7edd1b9700 1 --2- 172.21.15.72:0/4121966308 >> v2:172.21.15.179:3300/0 conn(0x7f7ed8066670 0x7f7ed8151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7edd1b9700 1 --2- 172.21.15.72:0/4121966308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf720 0x7f7ed8066180 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7edd1b9700 1 -- 172.21.15.72:0/4121966308 >> 172.21.15.72:0/4121966308 conn(0x7f7ed805ddc0 msgr2=0x7f7ed805e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:33.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7edd1b9700 1 -- 172.21.15.72:0/4121966308 shutdown_connections 2023-10-18T17:58:33.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.030+0000 7f7edd1b9700 1 -- 172.21.15.72:0/4121966308 wait complete. 2023-10-18T17:58:33.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edd1b9700 1 Processor -- start 2023-10-18T17:58:33.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edd1b9700 1 -- start start 2023-10-18T17:58:33.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edd1b9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed8066670 0x7f7ed80b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edd1b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf340 0x7f7ed8160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edd1b9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed80bf720 0x7f7ed814d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edd1b9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed814fce0 con 0x7f7ed80bf340 2023-10-18T17:58:33.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edd1b9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8160ef0 con 0x7f7ed80bf720 2023-10-18T17:58:33.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edd1b9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed80b8090 con 0x7f7ed8066670 2023-10-18T17:58:33.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed77fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf340 0x7f7ed8160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed77fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf340 0x7f7ed8160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36440/0 (socket says 172.21.15.72:36440) 2023-10-18T17:58:33.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed77fe700 1 -- 172.21.15.72:0/2600581828 learned_addr learned my addr 172.21.15.72:0/2600581828 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:33.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed7fff700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed8066670 0x7f7ed80b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edc9b8700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed80bf720 0x7f7ed814d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed77fe700 1 -- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed8066670 msgr2=0x7f7ed80b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed77fe700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed8066670 0x7f7ed80b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed77fe700 1 -- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed80bf720 msgr2=0x7f7ed814d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed77fe700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed80bf720 0x7f7ed814d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed77fe700 1 -- 172.21.15.72:0/2600581828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00092a0 con 0x7f7ed80bf340 2023-10-18T17:58:33.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edc9b8700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed80bf720 0x7f7ed814d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:33.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ed77fe700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf340 0x7f7ed8160a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7ec0009dd0 tx=0x7f7ec0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:33.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ecffff700 1 -- 172.21.15.72:0/2600581828 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec000f7e0 con 0x7f7ed80bf340 2023-10-18T17:58:33.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ecffff700 1 -- 172.21.15.72:0/2600581828 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec000f940 con 0x7f7ed80bf340 2023-10-18T17:58:33.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7ecffff700 1 -- 172.21.15.72:0/2600581828 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec001b690 con 0x7f7ed80bf340 2023-10-18T17:58:33.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.034+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed814f0d0 con 0x7f7ed80bf340 2023-10-18T17:58:33.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.038+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed814f690 con 0x7f7ed80bf340 2023-10-18T17:58:33.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.038+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed814fe90 con 0x7f7ed80bf340 2023-10-18T17:58:33.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.038+0000 7f7ecffff700 1 -- 172.21.15.72:0/2600581828 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7ec001ba00 con 0x7f7ed80bf340 2023-10-18T17:58:33.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.038+0000 7f7ecffff700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7eb805a4e0 0x7f7eb805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.042+0000 7f7ecffff700 1 -- 172.21.15.72:0/2600581828 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7ec0075f30 con 0x7f7ed80bf340 2023-10-18T17:58:33.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.042+0000 7f7ed7fff700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7eb805a4e0 0x7f7eb805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.042+0000 7f7ed7fff700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7eb805a4e0 0x7f7eb805c9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f7ec80011e0 tx=0x7f7ec8007bd0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:33.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.046+0000 7f7ecffff700 1 -- 172.21.15.72:0/2600581828 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f7ed814fe90 con 0x7f7ed80bf340 2023-10-18T17:58:33.219 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.214+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7ed814fe90 con 0x7f7ed80bf340 2023-10-18T17:58:33.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.214+0000 7f7ecffff700 1 -- 172.21.15.72:0/2600581828 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ed814fe90 con 0x7f7ed80bf340 2023-10-18T17:58:33.220 INFO:teuthology.orchestra.run.smithi072.stdout:60129542151 2023-10-18T17:58:33.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7eb805a4e0 msgr2=0x7f7eb805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7eb805a4e0 0x7f7eb805c9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f7ec80011e0 tx=0x7f7ec8007bd0).stop 2023-10-18T17:58:33.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf340 msgr2=0x7f7ed8160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf340 0x7f7ed8160a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7ec0009dd0 tx=0x7f7ec0017000).stop 2023-10-18T17:58:33.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 shutdown_connections 2023-10-18T17:58:33.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7eb805a4e0 0x7f7eb805c9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed80bf720 0x7f7ed814d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7ed80bf340 0x7f7ed8160a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 --2- 172.21.15.72:0/2600581828 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f7ed8066670 0x7f7ed80b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 >> 172.21.15.72:0/2600581828 conn(0x7f7ed805ddc0 msgr2=0x7f7ed805e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:33.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 shutdown_connections 2023-10-18T17:58:33.225 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.218+0000 7f7edd1b9700 1 -- 172.21.15.72:0/2600581828 wait complete. 2023-10-18T17:58:33.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2023-10-18T17:58:33.242 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-18T17:58:33.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.402+0000 7f977e4f1700 1 Processor -- start 2023-10-18T17:58:33.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.402+0000 7f977e4f1700 1 -- start start 2023-10-18T17:58:33.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f977e4f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f977814ff00 0x7f97780c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f977e4f1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9778068200 0x7f9778068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f977e4f1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9778151600 con 0x7f97780c6600 2023-10-18T17:58:33.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f977e4f1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9778150ed0 con 0x7f9778068200 2023-10-18T17:58:33.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f977e4f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9778150390 con 0x7f977814ff00 2023-10-18T17:58:33.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f976ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f977814ff00 0x7f97780c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f976ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f977814ff00 0x7f97780c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59316/0 (socket says 172.21.15.72:59316) 2023-10-18T17:58:33.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f976ffff700 1 -- 172.21.15.72:0/3723328617 learned_addr learned my addr 172.21.15.72:0/3723328617 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:33.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f977ca8e700 1 --2- 172.21.15.72:0/3723328617 >> v2:172.21.15.179:3300/0 conn(0x7f9778068200 0x7f9778068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f9777fff700 1 --1- 172.21.15.72:0/3723328617 >> v1:172.21.15.72:6789/0 conn(0x7f97780c6600 0x7f9778062860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51144/0 (socket says 172.21.15.72:51144) 2023-10-18T17:58:33.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f976ffff700 1 -- 172.21.15.72:0/3723328617 >> v2:172.21.15.179:3300/0 conn(0x7f9778068200 msgr2=0x7f9778068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:33.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f976ffff700 1 --2- 172.21.15.72:0/3723328617 >> v2:172.21.15.179:3300/0 conn(0x7f9778068200 0x7f9778068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f976ffff700 1 -- 172.21.15.72:0/3723328617 >> v1:172.21.15.72:6789/0 conn(0x7f97780c6600 legacy=0x7f9778062860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f976ffff700 1 -- 172.21.15.72:0/3723328617 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9778150630 con 0x7f977814ff00 2023-10-18T17:58:33.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f976ffff700 1 --2- 172.21.15.72:0/3723328617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f977814ff00 0x7f97780c69e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9760006fd0 tx=0x7f97600090f0).ready entity=mon.2 client_cookie=534bf338f16bd584 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:33.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f97777fe700 1 -- 172.21.15.72:0/3723328617 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976000b610 con 0x7f977814ff00 2023-10-18T17:58:33.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f97777fe700 1 -- 172.21.15.72:0/3723328617 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976000b770 con 0x7f977814ff00 2023-10-18T17:58:33.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.406+0000 7f97777fe700 1 -- 172.21.15.72:0/3723328617 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976000baf0 con 0x7f977814ff00 2023-10-18T17:58:33.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.410+0000 7f977e4f1700 1 -- 172.21.15.72:0/3723328617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f977814ff00 msgr2=0x7f97780c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.410+0000 7f977e4f1700 1 --2- 172.21.15.72:0/3723328617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f977814ff00 0x7f97780c69e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9760006fd0 tx=0x7f97600090f0).stop 2023-10-18T17:58:33.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.410+0000 7f977e4f1700 1 -- 172.21.15.72:0/3723328617 shutdown_connections 2023-10-18T17:58:33.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.410+0000 7f977e4f1700 1 --2- 172.21.15.72:0/3723328617 >> v2:172.21.15.179:3300/0 conn(0x7f9778068200 0x7f9778068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.410+0000 7f977e4f1700 1 --2- 172.21.15.72:0/3723328617 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f977814ff00 0x7f97780c69e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.410+0000 7f977e4f1700 1 -- 172.21.15.72:0/3723328617 >> 172.21.15.72:0/3723328617 conn(0x7f9778065150 msgr2=0x7f977805dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:33.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.410+0000 7f977e4f1700 1 -- 172.21.15.72:0/3723328617 shutdown_connections 2023-10-18T17:58:33.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.410+0000 7f977e4f1700 1 -- 172.21.15.72:0/3723328617 wait complete. 2023-10-18T17:58:33.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.410+0000 7f977e4f1700 1 Processor -- start 2023-10-18T17:58:33.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977e4f1700 1 -- start start 2023-10-18T17:58:33.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977e4f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9778068200 0x7f977814d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977e4f1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97780c6600 0x7f97780b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977e4f1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97780b9ae0 0x7f97780b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977e4f1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780b9f60 con 0x7f97780b9ae0 2023-10-18T17:58:33.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977e4f1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780b3cc0 con 0x7f97780c6600 2023-10-18T17:58:33.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977e4f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97780bf550 con 0x7f9778068200 2023-10-18T17:58:33.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977ca8e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97780b9ae0 0x7f97780b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977ca8e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97780b9ae0 0x7f97780b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36450/0 (socket says 172.21.15.72:36450) 2023-10-18T17:58:33.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977ca8e700 1 -- 172.21.15.72:0/2569651697 learned_addr learned my addr 172.21.15.72:0/2569651697 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:33.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f9777fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9778068200 0x7f977814d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f976ffff700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97780c6600 0x7f97780b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977ca8e700 1 -- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9778068200 msgr2=0x7f977814d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977ca8e700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9778068200 0x7f977814d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977ca8e700 1 -- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97780c6600 msgr2=0x7f97780b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977ca8e700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97780c6600 0x7f97780b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977ca8e700 1 -- 172.21.15.72:0/2569651697 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9760009370 con 0x7f97780b9ae0 2023-10-18T17:58:33.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f976ffff700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97780c6600 0x7f97780b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:33.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.414+0000 7f977ca8e700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97780b9ae0 0x7f97780b4150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f9770009df0 tx=0x7f977000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:33.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.418+0000 7f9774ff9700 1 -- 172.21.15.72:0/2569651697 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977000cb50 con 0x7f97780b9ae0 2023-10-18T17:58:33.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.418+0000 7f9774ff9700 1 -- 172.21.15.72:0/2569651697 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977000f040 con 0x7f97780b9ae0 2023-10-18T17:58:33.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.418+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97780bf830 con 0x7f97780b9ae0 2023-10-18T17:58:33.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.418+0000 7f9774ff9700 1 -- 172.21.15.72:0/2569651697 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977000cec0 con 0x7f97780b9ae0 2023-10-18T17:58:33.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.418+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97780bfdf0 con 0x7f97780b9ae0 2023-10-18T17:58:33.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.418+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977806c190 con 0x7f97780b9ae0 2023-10-18T17:58:33.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.418+0000 7f9774ff9700 1 -- 172.21.15.72:0/2569651697 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9770013610 con 0x7f97780b9ae0 2023-10-18T17:58:33.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.422+0000 7f9774ff9700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f975805a540 0x7f975805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.422+0000 7f9774ff9700 1 -- 172.21.15.72:0/2569651697 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9770074f40 con 0x7f97780b9ae0 2023-10-18T17:58:33.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.422+0000 7f9777fff700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f975805a540 0x7f975805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.422+0000 7f9777fff700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f975805a540 0x7f975805ca00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9768007fd0 tx=0x7f9768007d30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:33.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.430+0000 7f9774ff9700 1 -- 172.21.15.72:0/2569651697 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f9770048720 con 0x7f97780b9ae0 2023-10-18T17:58:33.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.590+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9778150910 con 0x7f97780b9ae0 2023-10-18T17:58:33.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.594+0000 7f9774ff9700 1 -- 172.21.15.72:0/2569651697 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f977004bd40 con 0x7f97780b9ae0 2023-10-18T17:58:33.597 INFO:teuthology.orchestra.run.smithi072.stdout:60129542151 2023-10-18T17:58:33.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f975805a540 msgr2=0x7f975805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f975805a540 0x7f975805ca00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9768007fd0 tx=0x7f9768007d30).stop 2023-10-18T17:58:33.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97780b9ae0 msgr2=0x7f97780b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97780b9ae0 0x7f97780b4150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f9770009df0 tx=0x7f977000b040).stop 2023-10-18T17:58:33.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 shutdown_connections 2023-10-18T17:58:33.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f975805a540 0x7f975805ca00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97780b9ae0 0x7f97780b4150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97780c6600 0x7f97780b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 --2- 172.21.15.72:0/2569651697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9778068200 0x7f977814d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 >> 172.21.15.72:0/2569651697 conn(0x7f9778065150 msgr2=0x7f9778152e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:33.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 shutdown_connections 2023-10-18T17:58:33.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.598+0000 7f977e4f1700 1 -- 172.21.15.72:0/2569651697 wait complete. 2023-10-18T17:58:33.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2023-10-18T17:58:33.623 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-18T17:58:33.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.782+0000 7f706ebdb700 1 Processor -- start 2023-10-18T17:58:33.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.782+0000 7f706ebdb700 1 -- start start 2023-10-18T17:58:33.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f706ebdb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f70680b2630 0x7f7068150980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f706ebdb700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7068150e70 con 0x7f70680c2460 2023-10-18T17:58:33.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f706ebdb700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7068153a10 con 0x7f70681534d0 2023-10-18T17:58:33.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f706ebdb700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70681517b0 con 0x7f70680b2630 2023-10-18T17:58:33.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f706d178700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f70681534d0 0x7f70680bb590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:33502/0 (socket says 172.21.15.72:33502) 2023-10-18T17:58:33.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f706d178700 1 -- 172.21.15.72:0/195142698 learned_addr learned my addr 172.21.15.72:0/195142698 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:33.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f706c977700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f70680c2460 0x7f70680b1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51146/0 (socket says 172.21.15.72:51146) 2023-10-18T17:58:33.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f7067fff700 1 --2- 172.21.15.72:0/195142698 >> v2:172.21.15.179:3300/0 conn(0x7f70680b2630 0x7f7068150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f70677fe700 1 -- 172.21.15.72:0/195142698 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176218148 0 0) 0x7f7068153a10 con 0x7f70681534d0 2023-10-18T17:58:33.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f7067fff700 1 -- 172.21.15.72:0/195142698 >> v1:172.21.15.179:6789/0 conn(0x7f70681534d0 legacy=0x7f70680bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f7067fff700 1 -- 172.21.15.72:0/195142698 >> v1:172.21.15.72:6789/0 conn(0x7f70680c2460 legacy=0x7f70680b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.793 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f7067fff700 1 -- 172.21.15.72:0/195142698 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068153cf0 con 0x7f70680b2630 2023-10-18T17:58:33.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.786+0000 7f7067fff700 1 --2- 172.21.15.72:0/195142698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70680b2630 0x7f7068150980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f7054009760 tx=0x7f7054009060).ready entity=mon.1 client_cookie=ffde60ca41d372a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:33.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f70677fe700 1 -- 172.21.15.72:0/195142698 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400f510 con 0x7f70680b2630 2023-10-18T17:58:33.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f70677fe700 1 -- 172.21.15.72:0/195142698 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400f670 con 0x7f70680b2630 2023-10-18T17:58:33.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f70677fe700 1 -- 172.21.15.72:0/195142698 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400f9f0 con 0x7f70680b2630 2023-10-18T17:58:33.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f706ebdb700 1 -- 172.21.15.72:0/195142698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70680b2630 msgr2=0x7f7068150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f706ebdb700 1 --2- 172.21.15.72:0/195142698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70680b2630 0x7f7068150980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f7054009760 tx=0x7f7054009060).stop 2023-10-18T17:58:33.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f706ebdb700 1 -- 172.21.15.72:0/195142698 shutdown_connections 2023-10-18T17:58:33.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f706ebdb700 1 --2- 172.21.15.72:0/195142698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70680b2630 0x7f7068150980 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f706ebdb700 1 -- 172.21.15.72:0/195142698 >> 172.21.15.72:0/195142698 conn(0x7f706805dc20 msgr2=0x7f706805e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:33.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f706ebdb700 1 -- 172.21.15.72:0/195142698 shutdown_connections 2023-10-18T17:58:33.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f706ebdb700 1 -- 172.21.15.72:0/195142698 wait complete. 2023-10-18T17:58:33.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.790+0000 7f706ebdb700 1 Processor -- start 2023-10-18T17:58:33.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706ebdb700 1 -- start start 2023-10-18T17:58:33.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706ebdb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70680b2630 0x7f7068164050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706ebdb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70680c2460 0x7f70680b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706ebdb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70681534d0 0x7f70680afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706ebdb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7068147870 con 0x7f70680b2630 2023-10-18T17:58:33.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706ebdb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70680b1de0 con 0x7f70681534d0 2023-10-18T17:58:33.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706ebdb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7068147060 con 0x7f70680c2460 2023-10-18T17:58:33.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706c977700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70680b2630 0x7f7068164050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706d178700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70681534d0 0x7f70680afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f7067fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70680c2460 0x7f70680b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f7067fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70680c2460 0x7f70680b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59334/0 (socket says 172.21.15.72:59334) 2023-10-18T17:58:33.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706d178700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70681534d0 0x7f70680afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:56344/0 (socket says 172.21.15.72:56344) 2023-10-18T17:58:33.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706c977700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70680b2630 0x7f7068164050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36456/0 (socket says 172.21.15.72:36456) 2023-10-18T17:58:33.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f7067fff700 1 -- 172.21.15.72:0/1689951233 learned_addr learned my addr 172.21.15.72:0/1689951233 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:33.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706d178700 1 -- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70680c2460 msgr2=0x7f70680b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706d178700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70680c2460 0x7f70680b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706d178700 1 -- 172.21.15.72:0/1689951233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70680b2630 msgr2=0x7f7068164050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706d178700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70680b2630 0x7f7068164050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706d178700 1 -- 172.21.15.72:0/1689951233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540092a0 con 0x7f70681534d0 2023-10-18T17:58:33.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706c977700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70680b2630 0x7f7068164050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:33.803 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706d178700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70681534d0 0x7f70680afa60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f706000c7e0 tx=0x7f7060007eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:33.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f7064ff9700 1 -- 172.21.15.72:0/1689951233 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060005d30 con 0x7f70681534d0 2023-10-18T17:58:33.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f7064ff9700 1 -- 172.21.15.72:0/1689951233 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060005e90 con 0x7f70681534d0 2023-10-18T17:58:33.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70680b1fd0 con 0x7f70681534d0 2023-10-18T17:58:33.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f7064ff9700 1 -- 172.21.15.72:0/1689951233 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060007360 con 0x7f70681534d0 2023-10-18T17:58:33.804 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.794+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70681472b0 con 0x7f70681534d0 2023-10-18T17:58:33.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.798+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706806c190 con 0x7f70681534d0 2023-10-18T17:58:33.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.798+0000 7f7064ff9700 1 -- 172.21.15.72:0/1689951233 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7060007710 con 0x7f70681534d0 2023-10-18T17:58:33.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.802+0000 7f7064ff9700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f704c05a510 0x7f704c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:33.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.802+0000 7f7064ff9700 1 -- 172.21.15.72:0/1689951233 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f70600747b0 con 0x7f70681534d0 2023-10-18T17:58:33.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.802+0000 7f706c977700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f704c05a510 0x7f704c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:33.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.802+0000 7f706c977700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f704c05a510 0x7f704c05c9d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f705c007fd0 tx=0x7f705c007d10).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:33.815 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.810+0000 7f7064ff9700 1 -- 172.21.15.72:0/1689951233 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f7060047e70 con 0x7f70681534d0 2023-10-18T17:58:33.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.978+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f70680b1bc0 con 0x7f70681534d0 2023-10-18T17:58:33.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.978+0000 7f7064ff9700 1 -- 172.21.15.72:0/1689951233 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f706004b490 con 0x7f70681534d0 2023-10-18T17:58:33.984 INFO:teuthology.orchestra.run.smithi072.stdout:60129542150 2023-10-18T17:58:33.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f704c05a510 msgr2=0x7f704c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f704c05a510 0x7f704c05c9d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f705c007fd0 tx=0x7f705c007d10).stop 2023-10-18T17:58:33.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70681534d0 msgr2=0x7f70680afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:33.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70681534d0 0x7f70680afa60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f706000c7e0 tx=0x7f7060007eb0).stop 2023-10-18T17:58:33.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 shutdown_connections 2023-10-18T17:58:33.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f704c05a510 0x7f704c05c9d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70681534d0 0x7f70680afa60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f70680c2460 0x7f70680b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 --2- 172.21.15.72:0/1689951233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70680b2630 0x7f7068164050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.982+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 >> 172.21.15.72:0/1689951233 conn(0x7f706805dc20 msgr2=0x7f706805e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:33.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.986+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 shutdown_connections 2023-10-18T17:58:33.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:33.986+0000 7f706ebdb700 1 -- 172.21.15.72:0/1689951233 wait complete. 2023-10-18T17:58:34.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2023-10-18T17:58:34.011 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-18T17:58:34.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.174+0000 7f3695769700 1 Processor -- start 2023-10-18T17:58:34.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.174+0000 7f3695769700 1 -- start start 2023-10-18T17:58:34.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f3695769700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900c6600 0x7f36900c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f3695769700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3690068200 0x7f3690068650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f3695769700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3690151600 con 0x7f369014ff00 2023-10-18T17:58:34.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f3695769700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3690150ed0 con 0x7f3690068200 2023-10-18T17:58:34.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f3695769700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3690150390 con 0x7f36900c6600 2023-10-18T17:58:34.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900c6600 0x7f36900c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368f7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3690068200 0x7f3690068650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f3687fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f369014ff00 0x7f3690062860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:33518/0 (socket says 172.21.15.72:33518) 2023-10-18T17:58:34.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f3687fff700 1 -- 172.21.15.72:0/1256778123 learned_addr learned my addr 172.21.15.72:0/1256778123 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:34.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368e7fc700 1 -- 172.21.15.72:0/1256778123 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492099415 0 0) 0x7f3690151600 con 0x7f369014ff00 2023-10-18T17:58:34.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368e7fc700 1 -- 172.21.15.72:0/1256778123 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f367c0033f0 con 0x7f369014ff00 2023-10-18T17:58:34.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368f7fe700 1 -- 172.21.15.72:0/1256778123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900c6600 msgr2=0x7f36900c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:34.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368f7fe700 1 --2- 172.21.15.72:0/1256778123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900c6600 0x7f36900c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368f7fe700 1 -- 172.21.15.72:0/1256778123 >> v1:172.21.15.179:6789/0 conn(0x7f369014ff00 legacy=0x7f3690062860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368f7fe700 1 -- 172.21.15.72:0/1256778123 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3690150630 con 0x7f3690068200 2023-10-18T17:58:34.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368f7fe700 1 --2- 172.21.15.72:0/1256778123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3690068200 0x7f3690068650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f3688009e60 tx=0x7f3688009780).ready entity=mon.1 client_cookie=2dde1c986ad6ffee server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:34.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368e7fc700 1 -- 172.21.15.72:0/1256778123 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368800e070 con 0x7f3690068200 2023-10-18T17:58:34.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368e7fc700 1 -- 172.21.15.72:0/1256778123 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368800bb40 con 0x7f3690068200 2023-10-18T17:58:34.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.178+0000 7f368e7fc700 1 -- 172.21.15.72:0/1256778123 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688012410 con 0x7f3690068200 2023-10-18T17:58:34.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.182+0000 7f3695769700 1 -- 172.21.15.72:0/1256778123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3690068200 msgr2=0x7f3690068650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.182+0000 7f3695769700 1 --2- 172.21.15.72:0/1256778123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3690068200 0x7f3690068650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f3688009e60 tx=0x7f3688009780).stop 2023-10-18T17:58:34.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.182+0000 7f3695769700 1 -- 172.21.15.72:0/1256778123 shutdown_connections 2023-10-18T17:58:34.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.182+0000 7f3695769700 1 --2- 172.21.15.72:0/1256778123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3690068200 0x7f3690068650 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.182+0000 7f3695769700 1 --2- 172.21.15.72:0/1256778123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900c6600 0x7f36900c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.182+0000 7f3695769700 1 -- 172.21.15.72:0/1256778123 >> 172.21.15.72:0/1256778123 conn(0x7f3690065150 msgr2=0x7f369005dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:34.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.182+0000 7f3695769700 1 -- 172.21.15.72:0/1256778123 shutdown_connections 2023-10-18T17:58:34.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.182+0000 7f3695769700 1 -- 172.21.15.72:0/1256778123 wait complete. 2023-10-18T17:58:34.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.182+0000 7f3695769700 1 Processor -- start 2023-10-18T17:58:34.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3695769700 1 -- start start 2023-10-18T17:58:34.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3695769700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3690068200 0x7f36900bb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3695769700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36900c6600 0x7f36900b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3695769700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900b6030 0x7f36900bbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3695769700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369014d940 con 0x7f3690068200 2023-10-18T17:58:34.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3695769700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3690163de0 con 0x7f36900c6600 2023-10-18T17:58:34.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3695769700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36901631d0 con 0x7f36900b6030 2023-10-18T17:58:34.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f368effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3690068200 0x7f36900bb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3687fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36900c6600 0x7f36900b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f368f7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900b6030 0x7f36900bbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3687fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36900c6600 0x7f36900b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:56366/0 (socket says 172.21.15.72:56366) 2023-10-18T17:58:34.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3687fff700 1 -- 172.21.15.72:0/2770829727 learned_addr learned my addr 172.21.15.72:0/2770829727 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:34.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f368effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3690068200 0x7f36900bb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36468/0 (socket says 172.21.15.72:36468) 2023-10-18T17:58:34.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3687fff700 1 -- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900b6030 msgr2=0x7f36900bbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3687fff700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900b6030 0x7f36900bbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3687fff700 1 -- 172.21.15.72:0/2770829727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3690068200 msgr2=0x7f36900bb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3687fff700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3690068200 0x7f36900bb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f3687fff700 1 -- 172.21.15.72:0/2770829727 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3688009a20 con 0x7f36900c6600 2023-10-18T17:58:34.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.186+0000 7f368effd700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3690068200 0x7f36900bb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:34.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.190+0000 7f3687fff700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36900c6600 0x7f36900b5b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f367800c410 tx=0x7f3678007c30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:34.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.190+0000 7f3686ffd700 1 -- 172.21.15.72:0/2770829727 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36780070d0 con 0x7f36900c6600 2023-10-18T17:58:34.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.190+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36901634b0 con 0x7f36900c6600 2023-10-18T17:58:34.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.190+0000 7f3686ffd700 1 -- 172.21.15.72:0/2770829727 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3678007230 con 0x7f36900c6600 2023-10-18T17:58:34.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.190+0000 7f3686ffd700 1 -- 172.21.15.72:0/2770829727 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36780075d0 con 0x7f36900c6600 2023-10-18T17:58:34.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.190+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3690163a70 con 0x7f36900c6600 2023-10-18T17:58:34.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.190+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3690163f20 con 0x7f36900c6600 2023-10-18T17:58:34.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.194+0000 7f3686ffd700 1 -- 172.21.15.72:0/2770829727 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f36780059b0 con 0x7f36900c6600 2023-10-18T17:58:34.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.194+0000 7f3686ffd700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f367005a510 0x7f367005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.194+0000 7f3686ffd700 1 -- 172.21.15.72:0/2770829727 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3678073f20 con 0x7f36900c6600 2023-10-18T17:58:34.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.194+0000 7f368effd700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f367005a510 0x7f367005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.194+0000 7f368effd700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f367005a510 0x7f367005c9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f3680011310 tx=0x7f3680009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:34.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.202+0000 7f3686ffd700 1 -- 172.21.15.72:0/2770829727 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f3690163f20 con 0x7f36900c6600 2023-10-18T17:58:34.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.374+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f369006c190 con 0x7f36900c6600 2023-10-18T17:58:34.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.374+0000 7f3686ffd700 1 -- 172.21.15.72:0/2770829727 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3678018030 con 0x7f36900c6600 2023-10-18T17:58:34.378 INFO:teuthology.orchestra.run.smithi072.stdout:60129542151 2023-10-18T17:58:34.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f367005a510 msgr2=0x7f367005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f367005a510 0x7f367005c9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f3680011310 tx=0x7f3680009060).stop 2023-10-18T17:58:34.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36900c6600 msgr2=0x7f36900b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36900c6600 0x7f36900b5b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f367800c410 tx=0x7f3678007c30).stop 2023-10-18T17:58:34.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 shutdown_connections 2023-10-18T17:58:34.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f367005a510 0x7f367005c9d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f36900b6030 0x7f36900bbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36900c6600 0x7f36900b5b40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 --2- 172.21.15.72:0/2770829727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3690068200 0x7f36900bb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 >> 172.21.15.72:0/2770829727 conn(0x7f3690065150 msgr2=0x7f3690151a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:34.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 shutdown_connections 2023-10-18T17:58:34.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.378+0000 7f3695769700 1 -- 172.21.15.72:0/2770829727 wait complete. 2023-10-18T17:58:34.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2023-10-18T17:58:34.402 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-18T17:58:34.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.554+0000 7fbad981b700 1 Processor -- start 2023-10-18T17:58:34.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.554+0000 7fbad981b700 1 -- start start 2023-10-18T17:58:34.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fbad414d330 0x7fbad40beed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad4150a20 con 0x7fbad4067e50 2023-10-18T17:58:34.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad40659c0 con 0x7fbad40beaf0 2023-10-18T17:58:34.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad4062a90 con 0x7fbad414d330 2023-10-18T17:58:34.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbaca7fc700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fbad4067e50 0x7fbad414cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51162/0 (socket says 172.21.15.72:51162) 2023-10-18T17:58:34.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbaca7fc700 1 -- 172.21.15.72:0/983345603 learned_addr learned my addr 172.21.15.72:0/983345603 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:34.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad37fe700 1 --2- 172.21.15.72:0/983345603 >> v2:172.21.15.72:3300/0 conn(0x7fbad414d330 0x7fbad40beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad27fc700 1 -- 172.21.15.72:0/983345603 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188471348 0 0) 0x7fbad40659c0 con 0x7fbad40beaf0 2023-10-18T17:58:34.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad27fc700 1 -- 172.21.15.72:0/983345603 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac00033f0 con 0x7fbad40beaf0 2023-10-18T17:58:34.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad27fc700 1 -- 172.21.15.72:0/983345603 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752610891 0 0) 0x7fbad4150a20 con 0x7fbad4067e50 2023-10-18T17:58:34.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad27fc700 1 -- 172.21.15.72:0/983345603 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbad40659c0 con 0x7fbad4067e50 2023-10-18T17:58:34.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad37fe700 1 -- 172.21.15.72:0/983345603 >> v1:172.21.15.179:6789/0 conn(0x7fbad40beaf0 legacy=0x7fbad4065e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad37fe700 1 -- 172.21.15.72:0/983345603 >> v1:172.21.15.72:6789/0 conn(0x7fbad4067e50 legacy=0x7fbad414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad37fe700 1 -- 172.21.15.72:0/983345603 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad414fea0 con 0x7fbad414d330 2023-10-18T17:58:34.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad37fe700 1 --2- 172.21.15.72:0/983345603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad414d330 0x7fbad40beed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fbacc009e60 tx=0x7fbacc009780).ready entity=mon.0 client_cookie=62e554a2f09b4c28 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:34.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad27fc700 1 -- 172.21.15.72:0/983345603 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00e070 con 0x7fbad414d330 2023-10-18T17:58:34.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad27fc700 1 -- 172.21.15.72:0/983345603 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc00bab0 con 0x7fbad414d330 2023-10-18T17:58:34.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad27fc700 1 -- 172.21.15.72:0/983345603 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00be30 con 0x7fbad414d330 2023-10-18T17:58:34.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 -- 172.21.15.72:0/983345603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad414d330 msgr2=0x7fbad40beed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 --2- 172.21.15.72:0/983345603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad414d330 0x7fbad40beed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fbacc009e60 tx=0x7fbacc009780).stop 2023-10-18T17:58:34.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 -- 172.21.15.72:0/983345603 shutdown_connections 2023-10-18T17:58:34.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 --2- 172.21.15.72:0/983345603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad414d330 0x7fbad40beed0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 -- 172.21.15.72:0/983345603 >> 172.21.15.72:0/983345603 conn(0x7fbad40645a0 msgr2=0x7fbad40649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:34.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 -- 172.21.15.72:0/983345603 shutdown_connections 2023-10-18T17:58:34.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.558+0000 7fbad981b700 1 -- 172.21.15.72:0/983345603 wait complete. 2023-10-18T17:58:34.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad981b700 1 Processor -- start 2023-10-18T17:58:34.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad981b700 1 -- start start 2023-10-18T17:58:34.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad981b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbad4067e50 0x7fbad40b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad981b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbad40beaf0 0x7fbad4163750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad981b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad4152280 0x7fbad40c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad981b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad4150ed0 con 0x7fbad4152280 2023-10-18T17:58:34.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad981b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad41632c0 con 0x7fbad40beaf0 2023-10-18T17:58:34.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad981b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad40b9db0 con 0x7fbad4067e50 2023-10-18T17:58:34.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad4152280 0x7fbad40c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad4152280 0x7fbad40c05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36484/0 (socket says 172.21.15.72:36484) 2023-10-18T17:58:34.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad37fe700 1 -- 172.21.15.72:0/3543889828 learned_addr learned my addr 172.21.15.72:0/3543889828 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:34.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbaca7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbad40beaf0 0x7fbad4163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.562+0000 7fbad2ffd700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbad4067e50 0x7fbad40b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbad37fe700 1 -- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbad4067e50 msgr2=0x7fbad40b98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbad37fe700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbad4067e50 0x7fbad40b98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbad37fe700 1 -- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbad40beaf0 msgr2=0x7fbad4163750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbad37fe700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbad40beaf0 0x7fbad4163750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbad37fe700 1 -- 172.21.15.72:0/3543889828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc009a20 con 0x7fbad4152280 2023-10-18T17:58:34.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbad37fe700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad4152280 0x7fbad40c05e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fbacc012cb0 tx=0x7fbad405edd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:34.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbacb7fe700 1 -- 172.21.15.72:0/3543889828 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc012e90 con 0x7fbad4152280 2023-10-18T17:58:34.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbacb7fe700 1 -- 172.21.15.72:0/3543889828 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc01d630 con 0x7fbad4152280 2023-10-18T17:58:34.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbacb7fe700 1 -- 172.21.15.72:0/3543889828 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc01d9b0 con 0x7fbad4152280 2023-10-18T17:58:34.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad4151120 con 0x7fbad4152280 2023-10-18T17:58:34.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad4162f50 con 0x7fbad4152280 2023-10-18T17:58:34.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad4046f00 con 0x7fbad4152280 2023-10-18T17:58:34.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.566+0000 7fbacb7fe700 1 -- 172.21.15.72:0/3543889828 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbacc01dd20 con 0x7fbad4152280 2023-10-18T17:58:34.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.570+0000 7fbacb7fe700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbab405a510 0x7fbab405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.570+0000 7fbacb7fe700 1 -- 172.21.15.72:0/3543889828 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbacc0764b0 con 0x7fbad4152280 2023-10-18T17:58:34.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.570+0000 7fbad2ffd700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbab405a510 0x7fbab405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.570+0000 7fbad2ffd700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbab405a510 0x7fbab405c9d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fbac4003930 tx=0x7fbac4007bc0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:34.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.574+0000 7fbacb7fe700 1 -- 172.21.15.72:0/3543889828 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fbacc049c90 con 0x7fbad4152280 2023-10-18T17:58:34.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.738+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbad4163400 con 0x7fbad4152280 2023-10-18T17:58:34.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.738+0000 7fbacb7fe700 1 -- 172.21.15.72:0/3543889828 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbacc04d2b0 con 0x7fbad4152280 2023-10-18T17:58:34.743 INFO:teuthology.orchestra.run.smithi072.stdout:55834574855 2023-10-18T17:58:34.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbab405a510 msgr2=0x7fbab405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbab405a510 0x7fbab405c9d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fbac4003930 tx=0x7fbac4007bc0).stop 2023-10-18T17:58:34.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad4152280 msgr2=0x7fbad40c05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad4152280 0x7fbad40c05e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fbacc012cb0 tx=0x7fbad405edd0).stop 2023-10-18T17:58:34.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 shutdown_connections 2023-10-18T17:58:34.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbab405a510 0x7fbab405c9d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbad4152280 0x7fbad40c05e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbad40beaf0 0x7fbad4163750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 --2- 172.21.15.72:0/3543889828 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbad4067e50 0x7fbad40b98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 >> 172.21.15.72:0/3543889828 conn(0x7fbad40645a0 msgr2=0x7fbad4064980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:34.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 shutdown_connections 2023-10-18T17:58:34.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.742+0000 7fbad981b700 1 -- 172.21.15.72:0/3543889828 wait complete. 2023-10-18T17:58:34.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.4 2023-10-18T17:58:34.766 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-18T17:58:34.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.926+0000 7f6fba28f700 1 Processor -- start 2023-10-18T17:58:34.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.926+0000 7f6fba28f700 1 -- start start 2023-10-18T17:58:34.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fba28f700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6fb4143400 0x7f6fb40bd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fba28f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb41437e0 0x7f6fb40bd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fba28f700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6fb414d750 0x7f6fb40bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fba28f700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb41501e0 con 0x7f6fb4143400 2023-10-18T17:58:34.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fba28f700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb41504c0 con 0x7f6fb414d750 2023-10-18T17:58:34.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fba28f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41507a0 con 0x7f6fb41437e0 2023-10-18T17:58:34.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb8a8c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb41437e0 0x7f6fb40bd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb9a8e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6fb414d750 0x7f6fb40bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb8a8c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb41437e0 0x7f6fb40bd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59372/0 (socket says 172.21.15.72:59372) 2023-10-18T17:58:34.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb8a8c700 1 -- 172.21.15.72:0/3508243224 learned_addr learned my addr 172.21.15.72:0/3508243224 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:34.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb928d700 1 --2- 172.21.15.72:0/3508243224 >> v2:172.21.15.72:3300/0 conn(0x7f6fb4143400 0x7f6fb40bd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb8a8c700 1 -- 172.21.15.72:0/3508243224 >> v2:172.21.15.179:3300/0 conn(0x7f6fb414d750 msgr2=0x7f6fb40bf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:34.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb8a8c700 1 --2- 172.21.15.72:0/3508243224 >> v2:172.21.15.179:3300/0 conn(0x7f6fb414d750 0x7f6fb40bf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb8a8c700 1 -- 172.21.15.72:0/3508243224 >> v2:172.21.15.72:3300/0 conn(0x7f6fb4143400 msgr2=0x7f6fb40bd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:34.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb8a8c700 1 --2- 172.21.15.72:0/3508243224 >> v2:172.21.15.72:3300/0 conn(0x7f6fb4143400 0x7f6fb40bd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb8a8c700 1 -- 172.21.15.72:0/3508243224 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4150a80 con 0x7f6fb41437e0 2023-10-18T17:58:34.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb9a8e700 1 --2- 172.21.15.72:0/3508243224 >> v2:172.21.15.179:3300/0 conn(0x7f6fb414d750 0x7f6fb40bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:34.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb928d700 1 --2- 172.21.15.72:0/3508243224 >> v2:172.21.15.72:3300/0 conn(0x7f6fb4143400 0x7f6fb40bd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:34.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fb8a8c700 1 --2- 172.21.15.72:0/3508243224 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb41437e0 0x7f6fb40bd7e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6fb40c3240 tx=0x7f6fa40090f0).ready entity=mon.2 client_cookie=32e3bd13bf969b48 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:34.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fabfff700 1 -- 172.21.15.72:0/3508243224 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400b5b0 con 0x7f6fb41437e0 2023-10-18T17:58:34.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fabfff700 1 -- 172.21.15.72:0/3508243224 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa400b710 con 0x7f6fb41437e0 2023-10-18T17:58:34.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.930+0000 7f6fabfff700 1 -- 172.21.15.72:0/3508243224 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400ba90 con 0x7f6fb41437e0 2023-10-18T17:58:34.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.934+0000 7f6fba28f700 1 -- 172.21.15.72:0/3508243224 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb41437e0 msgr2=0x7f6fb40bd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.934+0000 7f6fba28f700 1 --2- 172.21.15.72:0/3508243224 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb41437e0 0x7f6fb40bd7e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6fb40c3240 tx=0x7f6fa40090f0).stop 2023-10-18T17:58:34.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.934+0000 7f6fba28f700 1 -- 172.21.15.72:0/3508243224 shutdown_connections 2023-10-18T17:58:34.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.934+0000 7f6fba28f700 1 --2- 172.21.15.72:0/3508243224 >> v2:172.21.15.179:3300/0 conn(0x7f6fb414d750 0x7f6fb40bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.934+0000 7f6fba28f700 1 --2- 172.21.15.72:0/3508243224 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb41437e0 0x7f6fb40bd7e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.934+0000 7f6fba28f700 1 --2- 172.21.15.72:0/3508243224 >> v2:172.21.15.72:3300/0 conn(0x7f6fb4143400 0x7f6fb40bd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.934+0000 7f6fba28f700 1 -- 172.21.15.72:0/3508243224 >> 172.21.15.72:0/3508243224 conn(0x7f6fb405ddc0 msgr2=0x7f6fb405e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:34.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.934+0000 7f6fba28f700 1 -- 172.21.15.72:0/3508243224 shutdown_connections 2023-10-18T17:58:34.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.934+0000 7f6fba28f700 1 -- 172.21.15.72:0/3508243224 wait complete. 2023-10-18T17:58:34.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fba28f700 1 Processor -- start 2023-10-18T17:58:34.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fba28f700 1 -- start start 2023-10-18T17:58:34.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fba28f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb4143400 0x7f6fb414fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fba28f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb41437e0 0x7f6fb414ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fba28f700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb414d750 0x7f6fb414b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fba28f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb414c150 con 0x7f6fb4143400 2023-10-18T17:58:34.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fba28f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb414aa40 con 0x7f6fb41437e0 2023-10-18T17:58:34.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fba28f700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4149df0 con 0x7f6fb414d750 2023-10-18T17:58:34.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb928d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb4143400 0x7f6fb414fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb928d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb4143400 0x7f6fb414fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36496/0 (socket says 172.21.15.72:36496) 2023-10-18T17:58:34.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb928d700 1 -- 172.21.15.72:0/2786152981 learned_addr learned my addr 172.21.15.72:0/2786152981 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:34.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb8a8c700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb41437e0 0x7f6fb414ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb9a8e700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb414d750 0x7f6fb414b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb928d700 1 -- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb414d750 msgr2=0x7f6fb414b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb928d700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb414d750 0x7f6fb414b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb928d700 1 -- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb41437e0 msgr2=0x7f6fb414ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:34.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb928d700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb41437e0 0x7f6fb414ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:34.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb928d700 1 -- 172.21.15.72:0/2786152981 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac009a20 con 0x7f6fb4143400 2023-10-18T17:58:34.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb8a8c700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb41437e0 0x7f6fb414ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:34.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.938+0000 7f6fb928d700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb4143400 0x7f6fb414fb60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f6fb000d860 tx=0x7f6fb0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:34.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.942+0000 7f6fa97fa700 1 -- 172.21.15.72:0/2786152981 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0011590 con 0x7f6fb4143400 2023-10-18T17:58:34.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.942+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa4009370 con 0x7f6fb4143400 2023-10-18T17:58:34.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.942+0000 7f6fa97fa700 1 -- 172.21.15.72:0/2786152981 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb00116f0 con 0x7f6fb4143400 2023-10-18T17:58:34.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.942+0000 7f6fa97fa700 1 -- 172.21.15.72:0/2786152981 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0011a70 con 0x7f6fb4143400 2023-10-18T17:58:34.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.942+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb4160dc0 con 0x7f6fb4143400 2023-10-18T17:58:34.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.942+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4046f00 con 0x7f6fb4143400 2023-10-18T17:58:34.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.946+0000 7f6fa97fa700 1 -- 172.21.15.72:0/2786152981 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6fb0012420 con 0x7f6fb4143400 2023-10-18T17:58:34.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.946+0000 7f6fa97fa700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6f9805a540 0x7f6f9805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:34.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.946+0000 7f6fa97fa700 1 -- 172.21.15.72:0/2786152981 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6fb00743b0 con 0x7f6fb4143400 2023-10-18T17:58:34.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.946+0000 7f6fb8a8c700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6f9805a540 0x7f6f9805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:34.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.946+0000 7f6fb8a8c700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6f9805a540 0x7f6f9805ca00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f6fb4143bc0 tx=0x7f6fa4017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:34.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:34.950+0000 7f6fa97fa700 1 -- 172.21.15.72:0/2786152981 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f6fb0047b30 con 0x7f6fb4143400 2023-10-18T17:58:35.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.110+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6fb414acb0 con 0x7f6fb4143400 2023-10-18T17:58:35.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.110+0000 7f6fa97fa700 1 -- 172.21.15.72:0/2786152981 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6fb004b150 con 0x7f6fb4143400 2023-10-18T17:58:35.116 INFO:teuthology.orchestra.run.smithi072.stdout:55834574855 2023-10-18T17:58:35.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.114+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6f9805a540 msgr2=0x7f6f9805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.114+0000 7f6fba28f700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6f9805a540 0x7f6f9805ca00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f6fb4143bc0 tx=0x7f6fa4017000).stop 2023-10-18T17:58:35.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.114+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb4143400 msgr2=0x7f6fb414fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.114+0000 7f6fba28f700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb4143400 0x7f6fb414fb60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f6fb000d860 tx=0x7f6fb0008040).stop 2023-10-18T17:58:35.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.118+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 shutdown_connections 2023-10-18T17:58:35.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.118+0000 7f6fba28f700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6f9805a540 0x7f6f9805ca00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.118+0000 7f6fba28f700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6fb414d750 0x7f6fb414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.118+0000 7f6fba28f700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb41437e0 0x7f6fb414ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.118+0000 7f6fba28f700 1 --2- 172.21.15.72:0/2786152981 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fb4143400 0x7f6fb414fb60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.118+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 >> 172.21.15.72:0/2786152981 conn(0x7f6fb405ddc0 msgr2=0x7f6fb40bf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:35.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.118+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 shutdown_connections 2023-10-18T17:58:35.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.118+0000 7f6fba28f700 1 -- 172.21.15.72:0/2786152981 wait complete. 2023-10-18T17:58:35.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.5 2023-10-18T17:58:35.142 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-18T17:58:35.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.290+0000 7fe0c1569700 1 Processor -- start 2023-10-18T17:58:35.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0c1569700 1 -- start start 2023-10-18T17:58:35.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0c1569700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fe0bc0b5890 0x7fe0bc0beed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0c1569700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc067e50 con 0x7fe0bc0beaf0 2023-10-18T17:58:35.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0c1569700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc065a30 con 0x7fe0bc062640 2023-10-18T17:58:35.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0c1569700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc0bd8c0 con 0x7fe0bc0b5890 2023-10-18T17:58:35.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0baffd700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fe0bc0beaf0 0x7fe0bc065ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51168/0 (socket says 172.21.15.72:51168) 2023-10-18T17:58:35.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0baffd700 1 -- 172.21.15.72:0/1988868966 learned_addr learned my addr 172.21.15.72:0/1988868966 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:35.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0bb7fe700 1 --2- 172.21.15.72:0/1988868966 >> v2:172.21.15.179:3300/0 conn(0x7fe0bc0b5890 0x7fe0bc0beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0ba7fc700 1 -- 172.21.15.72:0/1988868966 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038639893 0 0) 0x7fe0bc067e50 con 0x7fe0bc0beaf0 2023-10-18T17:58:35.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0ba7fc700 1 -- 172.21.15.72:0/1988868966 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe09c0033f0 con 0x7fe0bc0beaf0 2023-10-18T17:58:35.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0bb7fe700 1 -- 172.21.15.72:0/1988868966 >> v1:172.21.15.179:6789/0 conn(0x7fe0bc062640 legacy=0x7fe0bc0b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0bb7fe700 1 -- 172.21.15.72:0/1988868966 >> v1:172.21.15.72:6789/0 conn(0x7fe0bc0beaf0 legacy=0x7fe0bc065ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0bb7fe700 1 -- 172.21.15.72:0/1988868966 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc1544f0 con 0x7fe0bc0b5890 2023-10-18T17:58:35.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.294+0000 7fe0bb7fe700 1 --2- 172.21.15.72:0/1988868966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0bc0b5890 0x7fe0bc0beed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe0b4009e60 tx=0x7fe0b4009780).ready entity=mon.1 client_cookie=9df47bd4c40c3823 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:35.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0ba7fc700 1 -- 172.21.15.72:0/1988868966 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400e070 con 0x7fe0bc0b5890 2023-10-18T17:58:35.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0ba7fc700 1 -- 172.21.15.72:0/1988868966 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b400bb60 con 0x7fe0bc0b5890 2023-10-18T17:58:35.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0ba7fc700 1 -- 172.21.15.72:0/1988868966 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4012410 con 0x7fe0bc0b5890 2023-10-18T17:58:35.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0c1569700 1 -- 172.21.15.72:0/1988868966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0bc0b5890 msgr2=0x7fe0bc0beed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0c1569700 1 --2- 172.21.15.72:0/1988868966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0bc0b5890 0x7fe0bc0beed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe0b4009e60 tx=0x7fe0b4009780).stop 2023-10-18T17:58:35.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0c1569700 1 -- 172.21.15.72:0/1988868966 shutdown_connections 2023-10-18T17:58:35.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0c1569700 1 --2- 172.21.15.72:0/1988868966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0bc0b5890 0x7fe0bc0beed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0c1569700 1 -- 172.21.15.72:0/1988868966 >> 172.21.15.72:0/1988868966 conn(0x7fe0bc0645a0 msgr2=0x7fe0bc0649b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:35.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0c1569700 1 -- 172.21.15.72:0/1988868966 shutdown_connections 2023-10-18T17:58:35.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0c1569700 1 -- 172.21.15.72:0/1988868966 wait complete. 2023-10-18T17:58:35.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.298+0000 7fe0c1569700 1 Processor -- start 2023-10-18T17:58:35.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0c1569700 1 -- start start 2023-10-18T17:58:35.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0c1569700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc062640 0x7fe0bc157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0c1569700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0bc0b5890 0x7fe0bc0b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0c1569700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0bc0beaf0 0x7fe0bc0bb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0c1569700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc155050 con 0x7fe0bc0b5890 2023-10-18T17:58:35.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0c1569700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc165110 con 0x7fe0bc0beaf0 2023-10-18T17:58:35.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0c1569700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0b7230 con 0x7fe0bc062640 2023-10-18T17:58:35.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b27fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0bc0b5890 0x7fe0bc0b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b27fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0bc0b5890 0x7fe0bc0b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36500/0 (socket says 172.21.15.72:36500) 2023-10-18T17:58:35.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b27fc700 1 -- 172.21.15.72:0/600561142 learned_addr learned my addr 172.21.15.72:0/600561142 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:35.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0bb7fe700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0bc0beaf0 0x7fe0bc0bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0baffd700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc062640 0x7fe0bc157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b27fc700 1 -- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc062640 msgr2=0x7fe0bc157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b27fc700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc062640 0x7fe0bc157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b27fc700 1 -- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0bc0beaf0 msgr2=0x7fe0bc0bb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b27fc700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0bc0beaf0 0x7fe0bc0bb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b27fc700 1 -- 172.21.15.72:0/600561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b4009a20 con 0x7fe0bc0b5890 2023-10-18T17:58:35.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0baffd700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc062640 0x7fe0bc157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:35.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b27fc700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0bc0b5890 0x7fe0bc0b5e60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe0a400c4c0 tx=0x7fe0a4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:35.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b37fe700 1 -- 172.21.15.72:0/600561142 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4007020 con 0x7fe0bc0b5890 2023-10-18T17:58:35.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b37fe700 1 -- 172.21.15.72:0/600561142 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a4001e90 con 0x7fe0bc0b5890 2023-10-18T17:58:35.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc159360 con 0x7fe0bc0b5890 2023-10-18T17:58:35.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0b37fe700 1 -- 172.21.15.72:0/600561142 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40073b0 con 0x7fe0bc0b5890 2023-10-18T17:58:35.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.302+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc158d10 con 0x7fe0bc0b5890 2023-10-18T17:58:35.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.306+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc158b50 con 0x7fe0bc0b5890 2023-10-18T17:58:35.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.306+0000 7fe0b37fe700 1 -- 172.21.15.72:0/600561142 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe0a40068b0 con 0x7fe0bc0b5890 2023-10-18T17:58:35.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.306+0000 7fe0b37fe700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe0a005a540 0x7fe0a005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.306+0000 7fe0b37fe700 1 -- 172.21.15.72:0/600561142 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe0a4074180 con 0x7fe0bc0b5890 2023-10-18T17:58:35.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.310+0000 7fe0baffd700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe0a005a540 0x7fe0a005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.310+0000 7fe0baffd700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe0a005a540 0x7fe0a005ca00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fe0ac007fd0 tx=0x7fe0ac009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:35.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.314+0000 7fe0b37fe700 1 -- 172.21.15.72:0/600561142 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fe0a40478a0 con 0x7fe0bc0b5890 2023-10-18T17:58:35.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.478+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe0bc0c1c60 con 0x7fe0bc0b5890 2023-10-18T17:58:35.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0b37fe700 1 -- 172.21.15.72:0/600561142 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0a404aec0 con 0x7fe0bc0b5890 2023-10-18T17:58:35.486 INFO:teuthology.orchestra.run.smithi072.stdout:55834574855 2023-10-18T17:58:35.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe0a005a540 msgr2=0x7fe0a005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe0a005a540 0x7fe0a005ca00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fe0ac007fd0 tx=0x7fe0ac009040).stop 2023-10-18T17:58:35.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0bc0b5890 msgr2=0x7fe0bc0b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0bc0b5890 0x7fe0bc0b5e60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe0a400c4c0 tx=0x7fe0a4007e00).stop 2023-10-18T17:58:35.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 shutdown_connections 2023-10-18T17:58:35.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe0a005a540 0x7fe0a005ca00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0bc0beaf0 0x7fe0bc0bb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0bc0b5890 0x7fe0bc0b5e60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 --2- 172.21.15.72:0/600561142 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe0bc062640 0x7fe0bc157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 >> 172.21.15.72:0/600561142 conn(0x7fe0bc0645a0 msgr2=0x7fe0bc153c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:35.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 shutdown_connections 2023-10-18T17:58:35.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.482+0000 7fe0c1569700 1 -- 172.21.15.72:0/600561142 wait complete. 2023-10-18T17:58:35.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.6 2023-10-18T17:58:35.507 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-18T17:58:35.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.670+0000 7f0a1de9b700 1 Processor -- start 2023-10-18T17:58:35.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.670+0000 7f0a1de9b700 1 -- start start 2023-10-18T17:58:35.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180c2460 0x7f0a18150980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a180b2630 0x7f0a180c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a18150e70 con 0x7f0a181534d0 2023-10-18T17:58:35.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a181517b0 con 0x7f0a180b2630 2023-10-18T17:58:35.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180c2d30 con 0x7f0a180c2460 2023-10-18T17:58:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a17fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a180b2630 0x7f0a180c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a177fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180c2460 0x7f0a18150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a17fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f0a180b2630 0x7f0a180c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:56430/0 (socket says 172.21.15.72:56430) 2023-10-18T17:58:35.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a17fff700 1 -- 172.21.15.72:0/3110318358 learned_addr learned my addr 172.21.15.72:0/3110318358 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:35.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a16ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f0a181534d0 0x7f0a180b1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:33538/0 (socket says 172.21.15.72:33538) 2023-10-18T17:58:35.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a17fff700 1 -- 172.21.15.72:0/3110318358 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180c2460 msgr2=0x7f0a18150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:35.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a17fff700 1 --2- 172.21.15.72:0/3110318358 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180c2460 0x7f0a18150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a17fff700 1 -- 172.21.15.72:0/3110318358 >> v1:172.21.15.179:6789/0 conn(0x7f0a181534d0 legacy=0x7f0a180b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a17fff700 1 -- 172.21.15.72:0/3110318358 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a18153a10 con 0x7f0a180b2630 2023-10-18T17:58:35.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a17fff700 1 --2- 172.21.15.72:0/3110318358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a180b2630 0x7f0a180c2840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0a10009f20 tx=0x7f0a10009840).ready entity=mon.1 client_cookie=ce9c3ade2b4c0297 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:35.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a167fc700 1 -- 172.21.15.72:0/3110318358 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000e070 con 0x7f0a180b2630 2023-10-18T17:58:35.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a167fc700 1 -- 172.21.15.72:0/3110318358 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1000ba60 con 0x7f0a180b2630 2023-10-18T17:58:35.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a167fc700 1 -- 172.21.15.72:0/3110318358 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000bde0 con 0x7f0a180b2630 2023-10-18T17:58:35.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 -- 172.21.15.72:0/3110318358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a180b2630 msgr2=0x7f0a180c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 --2- 172.21.15.72:0/3110318358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a180b2630 0x7f0a180c2840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0a10009f20 tx=0x7f0a10009840).stop 2023-10-18T17:58:35.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 -- 172.21.15.72:0/3110318358 shutdown_connections 2023-10-18T17:58:35.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 --2- 172.21.15.72:0/3110318358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a180b2630 0x7f0a180c2840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 --2- 172.21.15.72:0/3110318358 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180c2460 0x7f0a18150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.674+0000 7f0a1de9b700 1 -- 172.21.15.72:0/3110318358 >> 172.21.15.72:0/3110318358 conn(0x7f0a1805dc20 msgr2=0x7f0a1805e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:35.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.678+0000 7f0a1de9b700 1 -- 172.21.15.72:0/3110318358 shutdown_connections 2023-10-18T17:58:35.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.678+0000 7f0a1de9b700 1 -- 172.21.15.72:0/3110318358 wait complete. 2023-10-18T17:58:35.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.678+0000 7f0a1de9b700 1 Processor -- start 2023-10-18T17:58:35.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.678+0000 7f0a1de9b700 1 -- start start 2023-10-18T17:58:35.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.678+0000 7f0a1de9b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180b2630 0x7f0a18160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a1de9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a180c2460 0x7f0a18154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a1de9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a181534d0 0x7f0a180af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a1de9b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180b57c0 con 0x7f0a181534d0 2023-10-18T17:58:35.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a1de9b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18156830 con 0x7f0a180c2460 2023-10-18T17:58:35.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a1de9b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a181610a0 con 0x7f0a180b2630 2023-10-18T17:58:35.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a17fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a181534d0 0x7f0a180af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a17fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a181534d0 0x7f0a180af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36514/0 (socket says 172.21.15.72:36514) 2023-10-18T17:58:35.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a17fff700 1 -- 172.21.15.72:0/2058261796 learned_addr learned my addr 172.21.15.72:0/2058261796 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:35.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a177fe700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180b2630 0x7f0a18160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a16ffd700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a180c2460 0x7f0a18154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a17fff700 1 -- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180b2630 msgr2=0x7f0a18160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a17fff700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180b2630 0x7f0a18160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a17fff700 1 -- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a180c2460 msgr2=0x7f0a18154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a17fff700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a180c2460 0x7f0a18154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a17fff700 1 -- 172.21.15.72:0/2058261796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a10009ae0 con 0x7f0a181534d0 2023-10-18T17:58:35.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a17fff700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a181534d0 0x7f0a180af620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f0a180661d0 tx=0x7f0a10009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:35.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f09ff7fe700 1 -- 172.21.15.72:0/2058261796 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10009500 con 0x7f0a181534d0 2023-10-18T17:58:35.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f09ff7fe700 1 -- 172.21.15.72:0/2058261796 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a100092e0 con 0x7f0a181534d0 2023-10-18T17:58:35.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f09ff7fe700 1 -- 172.21.15.72:0/2058261796 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1001e6c0 con 0x7f0a181534d0 2023-10-18T17:58:35.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a180b00d0 con 0x7f0a181534d0 2023-10-18T17:58:35.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a181561e0 con 0x7f0a181534d0 2023-10-18T17:58:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.682+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a18156020 con 0x7f0a181534d0 2023-10-18T17:58:35.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.686+0000 7f09ff7fe700 1 -- 172.21.15.72:0/2058261796 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f0a10003040 con 0x7f0a181534d0 2023-10-18T17:58:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.686+0000 7f09ff7fe700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f0a0405a510 0x7f0a0405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.686+0000 7f09ff7fe700 1 -- 172.21.15.72:0/2058261796 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0a10075b90 con 0x7f0a181534d0 2023-10-18T17:58:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.686+0000 7f0a177fe700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f0a0405a510 0x7f0a0405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:35.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.686+0000 7f0a177fe700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f0a0405a510 0x7f0a0405c9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0a08009c30 tx=0x7f0a08009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:35.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.694+0000 7f09ff7fe700 1 -- 172.21.15.72:0/2058261796 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f0a10049360 con 0x7f0a181534d0 2023-10-18T17:58:35.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.858+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0a1806c190 con 0x7f0a181534d0 2023-10-18T17:58:35.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.858+0000 7f09ff7fe700 1 -- 172.21.15.72:0/2058261796 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a10023030 con 0x7f0a181534d0 2023-10-18T17:58:35.862 INFO:teuthology.orchestra.run.smithi072.stdout:55834574854 2023-10-18T17:58:35.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f0a0405a510 msgr2=0x7f0a0405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f0a0405a510 0x7f0a0405c9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0a08009c30 tx=0x7f0a08009060).stop 2023-10-18T17:58:35.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a181534d0 msgr2=0x7f0a180af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:35.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a181534d0 0x7f0a180af620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f0a180661d0 tx=0x7f0a10009840).stop 2023-10-18T17:58:35.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 shutdown_connections 2023-10-18T17:58:35.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f0a0405a510 0x7f0a0405c9d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a181534d0 0x7f0a180af620 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a180c2460 0x7f0a18154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 --2- 172.21.15.72:0/2058261796 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0a180b2630 0x7f0a18160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:35.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 >> 172.21.15.72:0/2058261796 conn(0x7f0a1805dc20 msgr2=0x7f0a180bfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:35.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 shutdown_connections 2023-10-18T17:58:35.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:35.862+0000 7f0a1de9b700 1 -- 172.21.15.72:0/2058261796 wait complete. 2023-10-18T17:58:35.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.7 2023-10-18T17:58:35.888 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-18T17:58:35.889 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T17:58:36.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.050+0000 7f9002d69700 1 Processor -- start 2023-10-18T17:58:36.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.050+0000 7f9002d69700 1 -- start start 2023-10-18T17:58:36.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9002d69700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8ffc0bbf10 0x7f8ffc0bb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9002d69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc0660c0 0x7f8ffc14e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9002d69700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ffc14e450 con 0x7f8ffc0bf700 2023-10-18T17:58:36.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9002d69700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ffc0b95f0 con 0x7f8ffc0bbf10 2023-10-18T17:58:36.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9002d69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc14de20 con 0x7f8ffc0660c0 2023-10-18T17:58:36.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9000b05700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8ffc0bbf10 0x7f8ffc0bb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9000b05700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8ffc0bbf10 0x7f8ffc0bb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36528/0 (socket says 172.21.15.72:36528) 2023-10-18T17:58:36.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9000b05700 1 -- 172.21.15.72:0/3231422221 learned_addr learned my addr 172.21.15.72:0/3231422221 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:36.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9001306700 1 --2- 172.21.15.72:0/3231422221 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc0660c0 0x7f8ffc14e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9000b05700 1 -- 172.21.15.72:0/3231422221 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc0660c0 msgr2=0x7f8ffc14e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:36.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9000b05700 1 --2- 172.21.15.72:0/3231422221 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc0660c0 0x7f8ffc14e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9000b05700 1 -- 172.21.15.72:0/3231422221 >> v1:172.21.15.72:6789/0 conn(0x7f8ffc0bf700 legacy=0x7f8ffc0beb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.054+0000 7f9000b05700 1 -- 172.21.15.72:0/3231422221 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc0b9890 con 0x7f8ffc0bbf10 2023-10-18T17:58:36.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9000b05700 1 --2- 172.21.15.72:0/3231422221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0bbf10 0x7f8ffc0bb310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f8ff0006fd0 tx=0x7f8ff00090f0).ready entity=mon.0 client_cookie=445c39cf743f3133 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:36.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f8ffb7fe700 1 -- 172.21.15.72:0/3231422221 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff000b580 con 0x7f8ffc0bbf10 2023-10-18T17:58:36.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f8ffb7fe700 1 -- 172.21.15.72:0/3231422221 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff000b6e0 con 0x7f8ffc0bbf10 2023-10-18T17:58:36.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f8ffb7fe700 1 -- 172.21.15.72:0/3231422221 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff000ba60 con 0x7f8ffc0bbf10 2023-10-18T17:58:36.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9002d69700 1 -- 172.21.15.72:0/3231422221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0bbf10 msgr2=0x7f8ffc0bb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9002d69700 1 --2- 172.21.15.72:0/3231422221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0bbf10 0x7f8ffc0bb310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f8ff0006fd0 tx=0x7f8ff00090f0).stop 2023-10-18T17:58:36.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9002d69700 1 -- 172.21.15.72:0/3231422221 shutdown_connections 2023-10-18T17:58:36.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9002d69700 1 --2- 172.21.15.72:0/3231422221 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc0660c0 0x7f8ffc14e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9002d69700 1 --2- 172.21.15.72:0/3231422221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0bbf10 0x7f8ffc0bb310 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9002d69700 1 -- 172.21.15.72:0/3231422221 >> 172.21.15.72:0/3231422221 conn(0x7f8ffc0620d0 msgr2=0x7f8ffc0624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:36.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9002d69700 1 -- 172.21.15.72:0/3231422221 shutdown_connections 2023-10-18T17:58:36.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9002d69700 1 -- 172.21.15.72:0/3231422221 wait complete. 2023-10-18T17:58:36.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.058+0000 7f9002d69700 1 Processor -- start 2023-10-18T17:58:36.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9002d69700 1 -- start start 2023-10-18T17:58:36.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9002d69700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0660c0 0x7f8ffc0b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9002d69700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ffc0bbf10 0x7f8ffc1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9002d69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc1518c0 0x7f8ffc0bff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9002d69700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc160fa0 con 0x7f8ffc0660c0 2023-10-18T17:58:36.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9002d69700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc161430 con 0x7f8ffc0bbf10 2023-10-18T17:58:36.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9002d69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0b6650 con 0x7f8ffc1518c0 2023-10-18T17:58:36.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9000b05700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0660c0 0x7f8ffc0b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9001306700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc1518c0 0x7f8ffc0bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9000b05700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0660c0 0x7f8ffc0b8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36542/0 (socket says 172.21.15.72:36542) 2023-10-18T17:58:36.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9000b05700 1 -- 172.21.15.72:0/2534200146 learned_addr learned my addr 172.21.15.72:0/2534200146 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:36.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f8ffbfff700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ffc0bbf10 0x7f8ffc1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9000b05700 1 -- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc1518c0 msgr2=0x7f8ffc0bff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9000b05700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc1518c0 0x7f8ffc0bff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9000b05700 1 -- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ffc0bbf10 msgr2=0x7f8ffc1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9000b05700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ffc0bbf10 0x7f8ffc1513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9000b05700 1 -- 172.21.15.72:0/2534200146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0009370 con 0x7f8ffc0660c0 2023-10-18T17:58:36.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f8ffbfff700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ffc0bbf10 0x7f8ffc1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:36.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9000b05700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0660c0 0x7f8ffc0b8660 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f8ff0009130 tx=0x7f8ff0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:36.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f8ff8ff9700 1 -- 172.21.15.72:0/2534200146 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff000b850 con 0x7f8ffc0660c0 2023-10-18T17:58:36.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f8ff8ff9700 1 -- 172.21.15.72:0/2534200146 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff000b9b0 con 0x7f8ffc0660c0 2023-10-18T17:58:36.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc0b6930 con 0x7f8ffc0660c0 2023-10-18T17:58:36.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f8ff8ff9700 1 -- 172.21.15.72:0/2534200146 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff001b660 con 0x7f8ffc0660c0 2023-10-18T17:58:36.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.062+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc160950 con 0x7f8ffc0660c0 2023-10-18T17:58:36.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.066+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ffc06c190 con 0x7f8ffc0660c0 2023-10-18T17:58:36.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.066+0000 7f8ff8ff9700 1 -- 172.21.15.72:0/2534200146 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8ff001ba10 con 0x7f8ffc0660c0 2023-10-18T17:58:36.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.066+0000 7f8ff8ff9700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8fe005a540 0x7f8fe005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.070+0000 7f8ff8ff9700 1 -- 172.21.15.72:0/2534200146 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8ff0076180 con 0x7f8ffc0660c0 2023-10-18T17:58:36.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.070+0000 7f8ffbfff700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8fe005a540 0x7f8fe005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.070+0000 7f8ffbfff700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8fe005a540 0x7f8fe005ca00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f8ffc0c2cc0 tx=0x7f8fe8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:36.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.074+0000 7f8ff8ff9700 1 -- 172.21.15.72:0/2534200146 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f8ff00498a0 con 0x7f8ffc0660c0 2023-10-18T17:58:36.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.230+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 --> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ffc066680 con 0x7f8fe005a540 2023-10-18T17:58:36.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.250+0000 7f8ff8ff9700 1 -- 172.21.15.72:0/2534200146 <== mgr.4102 v2:172.21.15.179:6832/33696 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336142 (secure 0 0 0) 0x7f8ffc066680 con 0x7f8fe005a540 2023-10-18T17:58:36.259 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:36.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.258+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8fe005a540 msgr2=0x7f8fe005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.258+0000 7f9002d69700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8fe005a540 0x7f8fe005ca00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f8ffc0c2cc0 tx=0x7f8fe8009040).stop 2023-10-18T17:58:36.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.258+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0660c0 msgr2=0x7f8ffc0b8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.258+0000 7f9002d69700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0660c0 0x7f8ffc0b8660 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f8ff0009130 tx=0x7f8ff0017000).stop 2023-10-18T17:58:36.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.262+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 shutdown_connections 2023-10-18T17:58:36.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.262+0000 7f9002d69700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8fe005a540 0x7f8fe005ca00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.262+0000 7f9002d69700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8ffc1518c0 0x7f8ffc0bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.262+0000 7f9002d69700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ffc0bbf10 0x7f8ffc1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.262+0000 7f9002d69700 1 --2- 172.21.15.72:0/2534200146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ffc0660c0 0x7f8ffc0b8660 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.262+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 >> 172.21.15.72:0/2534200146 conn(0x7f8ffc0620d0 msgr2=0x7f8ffc0bdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:36.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.262+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 shutdown_connections 2023-10-18T17:58:36.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.262+0000 7f9002d69700 1 -- 172.21.15.72:0/2534200146 wait complete. 2023-10-18T17:58:36.272 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-10-18T17:58:36.286 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-18T17:58:34.470559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378528,"kb_used_data":1760,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596192,"statfs":{"total":773094113280,"available":770658500608,"internally_reserved":0,"allocated":1802240,"data_stored":677080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129427+0000","last_change":"2023-10-18T17:58:23.125789+0000","last_active":"2023-10-18T17:58:23.129427+0000","last_peered":"2023-10-18T17:58:23.129427+0000","last_clean":"2023-10-18T17:58:23.129427+0000","last_became_active":"2023-10-18T17:58:23.123952+0000","last_became_peered":"2023-10-18T17:58:23.123952+0000","last_unstale":"2023-10-18T17:58:23.129427+0000","last_undegraded":"2023-10-18T17:58:23.129427+0000","last_fullsized":"2023-10-18T17:58:23.129427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103779+0000","last_change":"2023-10-18T17:58:22.680518+0000","last_active":"2023-10-18T17:58:23.103779+0000","last_peered":"2023-10-18T17:58:23.103779+0000","last_clean":"2023-10-18T17:58:23.103779+0000","last_became_active":"2023-10-18T17:58:22.679123+0000","last_became_peered":"2023-10-18T17:58:22.679123+0000","last_unstale":"2023-10-18T17:58:23.103779+0000","last_undegraded":"2023-10-18T17:58:23.103779+0000","last_fullsized":"2023-10-18T17:58:23.103779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129023+0000","last_change":"2023-10-18T17:58:23.121498+0000","last_active":"2023-10-18T17:58:23.129023+0000","last_peered":"2023-10-18T17:58:23.129023+0000","last_clean":"2023-10-18T17:58:23.129023+0000","last_became_active":"2023-10-18T17:58:23.121129+0000","last_became_peered":"2023-10-18T17:58:23.121129+0000","last_unstale":"2023-10-18T17:58:23.129023+0000","last_undegraded":"2023-10-18T17:58:23.129023+0000","last_fullsized":"2023-10-18T17:58:23.129023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114330+0000","last_change":"2023-10-18T17:58:22.701312+0000","last_active":"2023-10-18T17:58:23.114330+0000","last_peered":"2023-10-18T17:58:23.114330+0000","last_clean":"2023-10-18T17:58:23.114330+0000","last_became_active":"2023-10-18T17:58:22.701013+0000","last_became_peered":"2023-10-18T17:58:22.701013+0000","last_unstale":"2023-10-18T17:58:23.114330+0000","last_undegraded":"2023-10-18T17:58:23.114330+0000","last_fullsized":"2023-10-18T17:58:23.114330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120622+0000","last_change":"2023-10-18T17:58:22.690786+0000","last_active":"2023-10-18T17:58:23.120622+0000","last_peered":"2023-10-18T17:58:23.120622+0000","last_clean":"2023-10-18T17:58:23.120622+0000","last_became_active":"2023-10-18T17:58:22.690530+0000","last_became_peered":"2023-10-18T17:58:22.690530+0000","last_unstale":"2023-10-18T17:58:23.120622+0000","last_undegraded":"2023-10-18T17:58:23.120622+0000","last_fullsized":"2023-10-18T17:58:23.120622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121219+0000","last_change":"2023-10-18T17:58:23.112227+0000","last_active":"2023-10-18T17:58:23.121219+0000","last_peered":"2023-10-18T17:58:23.121219+0000","last_clean":"2023-10-18T17:58:23.121219+0000","last_became_active":"2023-10-18T17:58:23.111236+0000","last_became_peered":"2023-10-18T17:58:23.111236+0000","last_unstale":"2023-10-18T17:58:23.121219+0000","last_undegraded":"2023-10-18T17:58:23.121219+0000","last_fullsized":"2023-10-18T17:58:23.121219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.109161+0000","last_change":"2023-10-18T17:58:22.700039+0000","last_active":"2023-10-18T17:58:23.109161+0000","last_peered":"2023-10-18T17:58:23.109161+0000","last_clean":"2023-10-18T17:58:23.109161+0000","last_became_active":"2023-10-18T17:58:22.699719+0000","last_became_peered":"2023-10-18T17:58:22.699719+0000","last_unstale":"2023-10-18T17:58:23.109161+0000","last_undegraded":"2023-10-18T17:58:23.109161+0000","last_fullsized":"2023-10-18T17:58:23.109161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128700+0000","last_change":"2023-10-18T17:58:23.116723+0000","last_active":"2023-10-18T17:58:23.128700+0000","last_peered":"2023-10-18T17:58:23.128700+0000","last_clean":"2023-10-18T17:58:23.128700+0000","last_became_active":"2023-10-18T17:58:23.116376+0000","last_became_peered":"2023-10-18T17:58:23.116376+0000","last_unstale":"2023-10-18T17:58:23.128700+0000","last_undegraded":"2023-10-18T17:58:23.128700+0000","last_fullsized":"2023-10-18T17:58:23.128700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120037+0000","last_change":"2023-10-18T17:58:22.680018+0000","last_active":"2023-10-18T17:58:23.120037+0000","last_peered":"2023-10-18T17:58:23.120037+0000","last_clean":"2023-10-18T17:58:23.120037+0000","last_became_active":"2023-10-18T17:58:22.679464+0000","last_became_peered":"2023-10-18T17:58:22.679464+0000","last_unstale":"2023-10-18T17:58:23.120037+0000","last_undegraded":"2023-10-18T17:58:23.120037+0000","last_fullsized":"2023-10-18T17:58:23.120037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119206+0000","last_change":"2023-10-18T17:58:23.115758+0000","last_active":"2023-10-18T17:58:23.119206+0000","last_peered":"2023-10-18T17:58:23.119206+0000","last_clean":"2023-10-18T17:58:23.119206+0000","last_became_active":"2023-10-18T17:58:23.115471+0000","last_became_peered":"2023-10-18T17:58:23.115471+0000","last_unstale":"2023-10-18T17:58:23.119206+0000","last_undegraded":"2023-10-18T17:58:23.119206+0000","last_fullsized":"2023-10-18T17:58:23.119206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110821+0000","last_change":"2023-10-18T17:58:22.686370+0000","last_active":"2023-10-18T17:58:23.110821+0000","last_peered":"2023-10-18T17:58:23.110821+0000","last_clean":"2023-10-18T17:58:23.110821+0000","last_became_active":"2023-10-18T17:58:22.686062+0000","last_became_peered":"2023-10-18T17:58:22.686062+0000","last_unstale":"2023-10-18T17:58:23.110821+0000","last_undegraded":"2023-10-18T17:58:23.110821+0000","last_fullsized":"2023-10-18T17:58:23.110821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125366+0000","last_change":"2023-10-18T17:58:23.118319+0000","last_active":"2023-10-18T17:58:23.125366+0000","last_peered":"2023-10-18T17:58:23.125366+0000","last_clean":"2023-10-18T17:58:23.125366+0000","last_became_active":"2023-10-18T17:58:23.118018+0000","last_became_peered":"2023-10-18T17:58:23.118018+0000","last_unstale":"2023-10-18T17:58:23.125366+0000","last_undegraded":"2023-10-18T17:58:23.125366+0000","last_fullsized":"2023-10-18T17:58:23.125366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103546+0000","last_change":"2023-10-18T17:58:22.697114+0000","last_active":"2023-10-18T17:58:23.103546+0000","last_peered":"2023-10-18T17:58:23.103546+0000","last_clean":"2023-10-18T17:58:23.103546+0000","last_became_active":"2023-10-18T17:58:22.696798+0000","last_became_peered":"2023-10-18T17:58:22.696798+0000","last_unstale":"2023-10-18T17:58:23.103546+0000","last_undegraded":"2023-10-18T17:58:23.103546+0000","last_fullsized":"2023-10-18T17:58:23.103546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125558+0000","last_change":"2023-10-18T17:58:23.117398+0000","last_active":"2023-10-18T17:58:23.125558+0000","last_peered":"2023-10-18T17:58:23.125558+0000","last_clean":"2023-10-18T17:58:23.125558+0000","last_became_active":"2023-10-18T17:58:23.117086+0000","last_became_peered":"2023-10-18T17:58:23.117086+0000","last_unstale":"2023-10-18T17:58:23.125558+0000","last_undegraded":"2023-10-18T17:58:23.125558+0000","last_fullsized":"2023-10-18T17:58:23.125558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103780+0000","last_change":"2023-10-18T17:58:22.678915+0000","last_active":"2023-10-18T17:58:23.103780+0000","last_peered":"2023-10-18T17:58:23.103780+0000","last_clean":"2023-10-18T17:58:23.103780+0000","last_became_active":"2023-10-18T17:58:22.678609+0000","last_became_peered":"2023-10-18T17:58:22.678609+0000","last_unstale":"2023-10-18T17:58:23.103780+0000","last_undegraded":"2023-10-18T17:58:23.103780+0000","last_fullsized":"2023-10-18T17:58:23.103780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125651+0000","last_change":"2023-10-18T17:58:23.122008+0000","last_active":"2023-10-18T17:58:23.125651+0000","last_peered":"2023-10-18T17:58:23.125651+0000","last_clean":"2023-10-18T17:58:23.125651+0000","last_became_active":"2023-10-18T17:58:23.121684+0000","last_became_peered":"2023-10-18T17:58:23.121684+0000","last_unstale":"2023-10-18T17:58:23.125651+0000","last_undegraded":"2023-10-18T17:58:23.125651+0000","last_fullsized":"2023-10-18T17:58:23.125651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103948+0000","last_change":"2023-10-18T17:58:22.668404+0000","last_active":"2023-10-18T17:58:23.103948+0000","last_peered":"2023-10-18T17:58:23.103948+0000","last_clean":"2023-10-18T17:58:23.103948+0000","last_became_active":"2023-10-18T17:58:22.668038+0000","last_became_peered":"2023-10-18T17:58:22.668038+0000","last_unstale":"2023-10-18T17:58:23.103948+0000","last_undegraded":"2023-10-18T17:58:23.103948+0000","last_fullsized":"2023-10-18T17:58:23.103948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127151+0000","last_change":"2023-10-18T17:58:23.121199+0000","last_active":"2023-10-18T17:58:23.127151+0000","last_peered":"2023-10-18T17:58:23.127151+0000","last_clean":"2023-10-18T17:58:23.127151+0000","last_became_active":"2023-10-18T17:58:23.120856+0000","last_became_peered":"2023-10-18T17:58:23.120856+0000","last_unstale":"2023-10-18T17:58:23.127151+0000","last_undegraded":"2023-10-18T17:58:23.127151+0000","last_fullsized":"2023-10-18T17:58:23.127151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114412+0000","last_change":"2023-10-18T17:58:22.688492+0000","last_active":"2023-10-18T17:58:23.114412+0000","last_peered":"2023-10-18T17:58:23.114412+0000","last_clean":"2023-10-18T17:58:23.114412+0000","last_became_active":"2023-10-18T17:58:22.688162+0000","last_became_peered":"2023-10-18T17:58:22.688162+0000","last_unstale":"2023-10-18T17:58:23.114412+0000","last_undegraded":"2023-10-18T17:58:23.114412+0000","last_fullsized":"2023-10-18T17:58:23.114412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128660+0000","last_change":"2023-10-18T17:58:23.125037+0000","last_active":"2023-10-18T17:58:23.128660+0000","last_peered":"2023-10-18T17:58:23.128660+0000","last_clean":"2023-10-18T17:58:23.128660+0000","last_became_active":"2023-10-18T17:58:23.124722+0000","last_became_peered":"2023-10-18T17:58:23.124722+0000","last_unstale":"2023-10-18T17:58:23.128660+0000","last_undegraded":"2023-10-18T17:58:23.128660+0000","last_fullsized":"2023-10-18T17:58:23.128660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103799+0000","last_change":"2023-10-18T17:58:22.680359+0000","last_active":"2023-10-18T17:58:23.103799+0000","last_peered":"2023-10-18T17:58:23.103799+0000","last_clean":"2023-10-18T17:58:23.103799+0000","last_became_active":"2023-10-18T17:58:22.679356+0000","last_became_peered":"2023-10-18T17:58:22.679356+0000","last_unstale":"2023-10-18T17:58:23.103799+0000","last_undegraded":"2023-10-18T17:58:23.103799+0000","last_fullsized":"2023-10-18T17:58:23.103799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129449+0000","last_change":"2023-10-18T17:58:23.125443+0000","last_active":"2023-10-18T17:58:23.129449+0000","last_peered":"2023-10-18T17:58:23.129449+0000","last_clean":"2023-10-18T17:58:23.129449+0000","last_became_active":"2023-10-18T17:58:23.124036+0000","last_became_peered":"2023-10-18T17:58:23.124036+0000","last_unstale":"2023-10-18T17:58:23.129449+0000","last_undegraded":"2023-10-18T17:58:23.129449+0000","last_fullsized":"2023-10-18T17:58:23.129449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116673+0000","last_change":"2023-10-18T17:58:22.695695+0000","last_active":"2023-10-18T17:58:23.116673+0000","last_peered":"2023-10-18T17:58:23.116673+0000","last_clean":"2023-10-18T17:58:23.116673+0000","last_became_active":"2023-10-18T17:58:22.694956+0000","last_became_peered":"2023-10-18T17:58:22.694956+0000","last_unstale":"2023-10-18T17:58:23.116673+0000","last_undegraded":"2023-10-18T17:58:23.116673+0000","last_fullsized":"2023-10-18T17:58:23.116673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127389+0000","last_change":"2023-10-18T17:58:23.124170+0000","last_active":"2023-10-18T17:58:23.127389+0000","last_peered":"2023-10-18T17:58:23.127389+0000","last_clean":"2023-10-18T17:58:23.127389+0000","last_became_active":"2023-10-18T17:58:23.123835+0000","last_became_peered":"2023-10-18T17:58:23.123835+0000","last_unstale":"2023-10-18T17:58:23.127389+0000","last_undegraded":"2023-10-18T17:58:23.127389+0000","last_fullsized":"2023-10-18T17:58:23.127389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118363+0000","last_change":"2023-10-18T17:58:22.678817+0000","last_active":"2023-10-18T17:58:23.118363+0000","last_peered":"2023-10-18T17:58:23.118363+0000","last_clean":"2023-10-18T17:58:23.118363+0000","last_became_active":"2023-10-18T17:58:22.678508+0000","last_became_peered":"2023-10-18T17:58:22.678508+0000","last_unstale":"2023-10-18T17:58:23.118363+0000","last_undegraded":"2023-10-18T17:58:23.118363+0000","last_fullsized":"2023-10-18T17:58:23.118363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.133265+0000","last_change":"2023-10-18T17:58:23.129445+0000","last_active":"2023-10-18T17:58:23.133265+0000","last_peered":"2023-10-18T17:58:23.133265+0000","last_clean":"2023-10-18T17:58:23.133265+0000","last_became_active":"2023-10-18T17:58:23.129038+0000","last_became_peered":"2023-10-18T17:58:23.129038+0000","last_unstale":"2023-10-18T17:58:23.133265+0000","last_undegraded":"2023-10-18T17:58:23.133265+0000","last_fullsized":"2023-10-18T17:58:23.133265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117910+0000","last_change":"2023-10-18T17:58:22.684469+0000","last_active":"2023-10-18T17:58:23.117910+0000","last_peered":"2023-10-18T17:58:23.117910+0000","last_clean":"2023-10-18T17:58:23.117910+0000","last_became_active":"2023-10-18T17:58:22.684160+0000","last_became_peered":"2023-10-18T17:58:22.684160+0000","last_unstale":"2023-10-18T17:58:23.117910+0000","last_undegraded":"2023-10-18T17:58:23.117910+0000","last_fullsized":"2023-10-18T17:58:23.117910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125948+0000","last_change":"2023-10-18T17:58:23.120664+0000","last_active":"2023-10-18T17:58:23.125948+0000","last_peered":"2023-10-18T17:58:23.125948+0000","last_clean":"2023-10-18T17:58:23.125948+0000","last_became_active":"2023-10-18T17:58:23.120359+0000","last_became_peered":"2023-10-18T17:58:23.120359+0000","last_unstale":"2023-10-18T17:58:23.125948+0000","last_undegraded":"2023-10-18T17:58:23.125948+0000","last_fullsized":"2023-10-18T17:58:23.125948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.105546+0000","last_change":"2023-10-18T17:58:22.676645+0000","last_active":"2023-10-18T17:58:23.105546+0000","last_peered":"2023-10-18T17:58:23.105546+0000","last_clean":"2023-10-18T17:58:23.105546+0000","last_became_active":"2023-10-18T17:58:22.676295+0000","last_became_peered":"2023-10-18T17:58:22.676295+0000","last_unstale":"2023-10-18T17:58:23.105546+0000","last_undegraded":"2023-10-18T17:58:23.105546+0000","last_fullsized":"2023-10-18T17:58:23.105546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127404+0000","last_change":"2023-10-18T17:58:23.124501+0000","last_active":"2023-10-18T17:58:23.127404+0000","last_peered":"2023-10-18T17:58:23.127404+0000","last_clean":"2023-10-18T17:58:23.127404+0000","last_became_active":"2023-10-18T17:58:23.124186+0000","last_became_peered":"2023-10-18T17:58:23.124186+0000","last_unstale":"2023-10-18T17:58:23.127404+0000","last_undegraded":"2023-10-18T17:58:23.127404+0000","last_fullsized":"2023-10-18T17:58:23.127404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117130+0000","last_change":"2023-10-18T17:58:22.690272+0000","last_active":"2023-10-18T17:58:23.117130+0000","last_peered":"2023-10-18T17:58:23.117130+0000","last_clean":"2023-10-18T17:58:23.117130+0000","last_became_active":"2023-10-18T17:58:22.689949+0000","last_became_peered":"2023-10-18T17:58:22.689949+0000","last_unstale":"2023-10-18T17:58:23.117130+0000","last_undegraded":"2023-10-18T17:58:23.117130+0000","last_fullsized":"2023-10-18T17:58:23.117130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134179+0000","last_change":"2023-10-18T17:58:23.126544+0000","last_active":"2023-10-18T17:58:23.134179+0000","last_peered":"2023-10-18T17:58:23.134179+0000","last_clean":"2023-10-18T17:58:23.134179+0000","last_became_active":"2023-10-18T17:58:23.125987+0000","last_became_peered":"2023-10-18T17:58:23.125987+0000","last_unstale":"2023-10-18T17:58:23.134179+0000","last_undegraded":"2023-10-18T17:58:23.134179+0000","last_fullsized":"2023-10-18T17:58:23.134179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.107900+0000","last_change":"2023-10-18T17:58:22.681429+0000","last_active":"2023-10-18T17:58:23.107900+0000","last_peered":"2023-10-18T17:58:23.107900+0000","last_clean":"2023-10-18T17:58:23.107900+0000","last_became_active":"2023-10-18T17:58:22.681145+0000","last_became_peered":"2023-10-18T17:58:22.681145+0000","last_unstale":"2023-10-18T17:58:23.107900+0000","last_undegraded":"2023-10-18T17:58:23.107900+0000","last_fullsized":"2023-10-18T17:58:23.107900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121294+0000","last_change":"2023-10-18T17:58:23.117504+0000","last_active":"2023-10-18T17:58:23.121294+0000","last_peered":"2023-10-18T17:58:23.121294+0000","last_clean":"2023-10-18T17:58:23.121294+0000","last_became_active":"2023-10-18T17:58:23.117192+0000","last_became_peered":"2023-10-18T17:58:23.117192+0000","last_unstale":"2023-10-18T17:58:23.121294+0000","last_undegraded":"2023-10-18T17:58:23.121294+0000","last_fullsized":"2023-10-18T17:58:23.121294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114324+0000","last_change":"2023-10-18T17:58:22.682889+0000","last_active":"2023-10-18T17:58:23.114324+0000","last_peered":"2023-10-18T17:58:23.114324+0000","last_clean":"2023-10-18T17:58:23.114324+0000","last_became_active":"2023-10-18T17:58:22.682273+0000","last_became_peered":"2023-10-18T17:58:22.682273+0000","last_unstale":"2023-10-18T17:58:23.114324+0000","last_undegraded":"2023-10-18T17:58:23.114324+0000","last_fullsized":"2023-10-18T17:58:23.114324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.132038+0000","last_change":"2023-10-18T17:58:23.126856+0000","last_active":"2023-10-18T17:58:23.132038+0000","last_peered":"2023-10-18T17:58:23.132038+0000","last_clean":"2023-10-18T17:58:23.132038+0000","last_became_active":"2023-10-18T17:58:23.126582+0000","last_became_peered":"2023-10-18T17:58:23.126582+0000","last_unstale":"2023-10-18T17:58:23.132038+0000","last_undegraded":"2023-10-18T17:58:23.132038+0000","last_fullsized":"2023-10-18T17:58:23.132038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104415+0000","last_change":"2023-10-18T17:58:22.680971+0000","last_active":"2023-10-18T17:58:23.104415+0000","last_peered":"2023-10-18T17:58:23.104415+0000","last_clean":"2023-10-18T17:58:23.104415+0000","last_became_active":"2023-10-18T17:58:22.678750+0000","last_became_peered":"2023-10-18T17:58:22.678750+0000","last_unstale":"2023-10-18T17:58:23.104415+0000","last_undegraded":"2023-10-18T17:58:23.104415+0000","last_fullsized":"2023-10-18T17:58:23.104415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135050+0000","last_change":"2023-10-18T17:58:23.130754+0000","last_active":"2023-10-18T17:58:23.135050+0000","last_peered":"2023-10-18T17:58:23.135050+0000","last_clean":"2023-10-18T17:58:23.135050+0000","last_became_active":"2023-10-18T17:58:23.130419+0000","last_became_peered":"2023-10-18T17:58:23.130419+0000","last_unstale":"2023-10-18T17:58:23.135050+0000","last_undegraded":"2023-10-18T17:58:23.135050+0000","last_fullsized":"2023-10-18T17:58:23.135050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104721+0000","last_change":"2023-10-18T17:58:22.683291+0000","last_active":"2023-10-18T17:58:23.104721+0000","last_peered":"2023-10-18T17:58:23.104721+0000","last_clean":"2023-10-18T17:58:23.104721+0000","last_became_active":"2023-10-18T17:58:22.682926+0000","last_became_peered":"2023-10-18T17:58:22.682926+0000","last_unstale":"2023-10-18T17:58:23.104721+0000","last_undegraded":"2023-10-18T17:58:23.104721+0000","last_fullsized":"2023-10-18T17:58:23.104721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128937+0000","last_change":"2023-10-18T17:58:23.126426+0000","last_active":"2023-10-18T17:58:23.128937+0000","last_peered":"2023-10-18T17:58:23.128937+0000","last_clean":"2023-10-18T17:58:23.128937+0000","last_became_active":"2023-10-18T17:58:23.126114+0000","last_became_peered":"2023-10-18T17:58:23.126114+0000","last_unstale":"2023-10-18T17:58:23.128937+0000","last_undegraded":"2023-10-18T17:58:23.128937+0000","last_fullsized":"2023-10-18T17:58:23.128937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119826+0000","last_change":"2023-10-18T17:58:22.690514+0000","last_active":"2023-10-18T17:58:23.119826+0000","last_peered":"2023-10-18T17:58:23.119826+0000","last_clean":"2023-10-18T17:58:23.119826+0000","last_became_active":"2023-10-18T17:58:22.689427+0000","last_became_peered":"2023-10-18T17:58:22.689427+0000","last_unstale":"2023-10-18T17:58:23.119826+0000","last_undegraded":"2023-10-18T17:58:23.119826+0000","last_fullsized":"2023-10-18T17:58:23.119826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135075+0000","last_change":"2023-10-18T17:58:23.128307+0000","last_active":"2023-10-18T17:58:23.135075+0000","last_peered":"2023-10-18T17:58:23.135075+0000","last_clean":"2023-10-18T17:58:23.135075+0000","last_became_active":"2023-10-18T17:58:23.127998+0000","last_became_peered":"2023-10-18T17:58:23.127998+0000","last_unstale":"2023-10-18T17:58:23.135075+0000","last_undegraded":"2023-10-18T17:58:23.135075+0000","last_fullsized":"2023-10-18T17:58:23.135075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104658+0000","last_change":"2023-10-18T17:58:22.686163+0000","last_active":"2023-10-18T17:58:23.104658+0000","last_peered":"2023-10-18T17:58:23.104658+0000","last_clean":"2023-10-18T17:58:23.104658+0000","last_became_active":"2023-10-18T17:58:22.685950+0000","last_became_peered":"2023-10-18T17:58:22.685950+0000","last_unstale":"2023-10-18T17:58:23.104658+0000","last_undegraded":"2023-10-18T17:58:23.104658+0000","last_fullsized":"2023-10-18T17:58:23.104658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125096+0000","last_change":"2023-10-18T17:58:23.123007+0000","last_active":"2023-10-18T17:58:23.125096+0000","last_peered":"2023-10-18T17:58:23.125096+0000","last_clean":"2023-10-18T17:58:23.125096+0000","last_became_active":"2023-10-18T17:58:23.122656+0000","last_became_peered":"2023-10-18T17:58:23.122656+0000","last_unstale":"2023-10-18T17:58:23.125096+0000","last_undegraded":"2023-10-18T17:58:23.125096+0000","last_fullsized":"2023-10-18T17:58:23.125096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103750+0000","last_change":"2023-10-18T17:58:22.690142+0000","last_active":"2023-10-18T17:58:23.103750+0000","last_peered":"2023-10-18T17:58:23.103750+0000","last_clean":"2023-10-18T17:58:23.103750+0000","last_became_active":"2023-10-18T17:58:22.689829+0000","last_became_peered":"2023-10-18T17:58:22.689829+0000","last_unstale":"2023-10-18T17:58:23.103750+0000","last_undegraded":"2023-10-18T17:58:23.103750+0000","last_fullsized":"2023-10-18T17:58:23.103750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.132030+0000","last_change":"2023-10-18T17:58:23.126572+0000","last_active":"2023-10-18T17:58:23.132030+0000","last_peered":"2023-10-18T17:58:23.132030+0000","last_clean":"2023-10-18T17:58:23.132030+0000","last_became_active":"2023-10-18T17:58:23.125662+0000","last_became_peered":"2023-10-18T17:58:23.125662+0000","last_unstale":"2023-10-18T17:58:23.132030+0000","last_undegraded":"2023-10-18T17:58:23.132030+0000","last_fullsized":"2023-10-18T17:58:23.132030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103952+0000","last_change":"2023-10-18T17:58:22.674168+0000","last_active":"2023-10-18T17:58:23.103952+0000","last_peered":"2023-10-18T17:58:23.103952+0000","last_clean":"2023-10-18T17:58:23.103952+0000","last_became_active":"2023-10-18T17:58:22.668382+0000","last_became_peered":"2023-10-18T17:58:22.668382+0000","last_unstale":"2023-10-18T17:58:23.103952+0000","last_undegraded":"2023-10-18T17:58:23.103952+0000","last_fullsized":"2023-10-18T17:58:23.103952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121427+0000","last_change":"2023-10-18T17:58:23.118550+0000","last_active":"2023-10-18T17:58:23.121427+0000","last_peered":"2023-10-18T17:58:23.121427+0000","last_clean":"2023-10-18T17:58:23.121427+0000","last_became_active":"2023-10-18T17:58:23.118162+0000","last_became_peered":"2023-10-18T17:58:23.118162+0000","last_unstale":"2023-10-18T17:58:23.121427+0000","last_undegraded":"2023-10-18T17:58:23.121427+0000","last_fullsized":"2023-10-18T17:58:23.121427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.105030+0000","last_change":"2023-10-18T17:58:22.678474+0000","last_active":"2023-10-18T17:58:23.105030+0000","last_peered":"2023-10-18T17:58:23.105030+0000","last_clean":"2023-10-18T17:58:23.105030+0000","last_became_active":"2023-10-18T17:58:22.678197+0000","last_became_peered":"2023-10-18T17:58:22.678197+0000","last_unstale":"2023-10-18T17:58:23.105030+0000","last_undegraded":"2023-10-18T17:58:23.105030+0000","last_fullsized":"2023-10-18T17:58:23.105030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128857+0000","last_change":"2023-10-18T17:58:23.120777+0000","last_active":"2023-10-18T17:58:23.128857+0000","last_peered":"2023-10-18T17:58:23.128857+0000","last_clean":"2023-10-18T17:58:23.128857+0000","last_became_active":"2023-10-18T17:58:23.116893+0000","last_became_peered":"2023-10-18T17:58:23.116893+0000","last_unstale":"2023-10-18T17:58:23.128857+0000","last_undegraded":"2023-10-18T17:58:23.128857+0000","last_fullsized":"2023-10-18T17:58:23.128857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104747+0000","last_change":"2023-10-18T17:58:22.677025+0000","last_active":"2023-10-18T17:58:23.104747+0000","last_peered":"2023-10-18T17:58:23.104747+0000","last_clean":"2023-10-18T17:58:23.104747+0000","last_became_active":"2023-10-18T17:58:22.676536+0000","last_became_peered":"2023-10-18T17:58:22.676536+0000","last_unstale":"2023-10-18T17:58:23.104747+0000","last_undegraded":"2023-10-18T17:58:23.104747+0000","last_fullsized":"2023-10-18T17:58:23.104747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129785+0000","last_change":"2023-10-18T17:58:23.121230+0000","last_active":"2023-10-18T17:58:23.129785+0000","last_peered":"2023-10-18T17:58:23.129785+0000","last_clean":"2023-10-18T17:58:23.129785+0000","last_became_active":"2023-10-18T17:58:23.120977+0000","last_became_peered":"2023-10-18T17:58:23.120977+0000","last_unstale":"2023-10-18T17:58:23.129785+0000","last_undegraded":"2023-10-18T17:58:23.129785+0000","last_fullsized":"2023-10-18T17:58:23.129785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.107108+0000","last_change":"2023-10-18T17:58:22.691513+0000","last_active":"2023-10-18T17:58:23.107108+0000","last_peered":"2023-10-18T17:58:23.107108+0000","last_clean":"2023-10-18T17:58:23.107108+0000","last_became_active":"2023-10-18T17:58:22.691220+0000","last_became_peered":"2023-10-18T17:58:22.691220+0000","last_unstale":"2023-10-18T17:58:23.107108+0000","last_undegraded":"2023-10-18T17:58:23.107108+0000","last_fullsized":"2023-10-18T17:58:23.107108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.131817+0000","last_change":"2023-10-18T17:58:23.128454+0000","last_active":"2023-10-18T17:58:23.131817+0000","last_peered":"2023-10-18T17:58:23.131817+0000","last_clean":"2023-10-18T17:58:23.131817+0000","last_became_active":"2023-10-18T17:58:23.128113+0000","last_became_peered":"2023-10-18T17:58:23.128113+0000","last_unstale":"2023-10-18T17:58:23.131817+0000","last_undegraded":"2023-10-18T17:58:23.131817+0000","last_fullsized":"2023-10-18T17:58:23.131817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116763+0000","last_change":"2023-10-18T17:58:22.695883+0000","last_active":"2023-10-18T17:58:23.116763+0000","last_peered":"2023-10-18T17:58:23.116763+0000","last_clean":"2023-10-18T17:58:23.116763+0000","last_became_active":"2023-10-18T17:58:22.695183+0000","last_became_peered":"2023-10-18T17:58:22.695183+0000","last_unstale":"2023-10-18T17:58:23.116763+0000","last_undegraded":"2023-10-18T17:58:23.116763+0000","last_fullsized":"2023-10-18T17:58:23.116763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.131910+0000","last_change":"2023-10-18T17:58:23.123828+0000","last_active":"2023-10-18T17:58:23.131910+0000","last_peered":"2023-10-18T17:58:23.131910+0000","last_clean":"2023-10-18T17:58:23.131910+0000","last_became_active":"2023-10-18T17:58:23.123552+0000","last_became_peered":"2023-10-18T17:58:23.123552+0000","last_unstale":"2023-10-18T17:58:23.131910+0000","last_undegraded":"2023-10-18T17:58:23.131910+0000","last_fullsized":"2023-10-18T17:58:23.131910+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111182+0000","last_change":"2023-10-18T17:58:22.685450+0000","last_active":"2023-10-18T17:58:23.111182+0000","last_peered":"2023-10-18T17:58:23.111182+0000","last_clean":"2023-10-18T17:58:23.111182+0000","last_became_active":"2023-10-18T17:58:22.685168+0000","last_became_peered":"2023-10-18T17:58:22.685168+0000","last_unstale":"2023-10-18T17:58:23.111182+0000","last_undegraded":"2023-10-18T17:58:23.111182+0000","last_fullsized":"2023-10-18T17:58:23.111182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120499+0000","last_change":"2023-10-18T17:58:23.114901+0000","last_active":"2023-10-18T17:58:23.120499+0000","last_peered":"2023-10-18T17:58:23.120499+0000","last_clean":"2023-10-18T17:58:23.120499+0000","last_became_active":"2023-10-18T17:58:23.114590+0000","last_became_peered":"2023-10-18T17:58:23.114590+0000","last_unstale":"2023-10-18T17:58:23.120499+0000","last_undegraded":"2023-10-18T17:58:23.120499+0000","last_fullsized":"2023-10-18T17:58:23.120499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112133+0000","last_change":"2023-10-18T17:58:22.692159+0000","last_active":"2023-10-18T17:58:23.112133+0000","last_peered":"2023-10-18T17:58:23.112133+0000","last_clean":"2023-10-18T17:58:23.112133+0000","last_became_active":"2023-10-18T17:58:22.691846+0000","last_became_peered":"2023-10-18T17:58:22.691846+0000","last_unstale":"2023-10-18T17:58:23.112133+0000","last_undegraded":"2023-10-18T17:58:23.112133+0000","last_fullsized":"2023-10-18T17:58:23.112133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136653+0000","last_change":"2023-10-18T17:58:23.134892+0000","last_active":"2023-10-18T17:58:23.136653+0000","last_peered":"2023-10-18T17:58:23.136653+0000","last_clean":"2023-10-18T17:58:23.136653+0000","last_became_active":"2023-10-18T17:58:23.134213+0000","last_became_peered":"2023-10-18T17:58:23.134213+0000","last_unstale":"2023-10-18T17:58:23.136653+0000","last_undegraded":"2023-10-18T17:58:23.136653+0000","last_fullsized":"2023-10-18T17:58:23.136653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120693+0000","last_change":"2023-10-18T17:58:17.791475+0000","last_active":"2023-10-18T17:58:23.120693+0000","last_peered":"2023-10-18T17:58:23.120693+0000","last_clean":"2023-10-18T17:58:23.120693+0000","last_became_active":"2023-10-18T17:58:17.790774+0000","last_became_peered":"2023-10-18T17:58:17.790774+0000","last_unstale":"2023-10-18T17:58:23.120693+0000","last_undegraded":"2023-10-18T17:58:23.120693+0000","last_fullsized":"2023-10-18T17:58:23.120693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104540+0000","last_change":"2023-10-18T17:58:22.676456+0000","last_active":"2023-10-18T17:58:23.104540+0000","last_peered":"2023-10-18T17:58:23.104540+0000","last_clean":"2023-10-18T17:58:23.104540+0000","last_became_active":"2023-10-18T17:58:22.676189+0000","last_became_peered":"2023-10-18T17:58:22.676189+0000","last_unstale":"2023-10-18T17:58:23.104540+0000","last_undegraded":"2023-10-18T17:58:23.104540+0000","last_fullsized":"2023-10-18T17:58:23.104540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136521+0000","last_change":"2023-10-18T17:58:23.133571+0000","last_active":"2023-10-18T17:58:23.136521+0000","last_peered":"2023-10-18T17:58:23.136521+0000","last_clean":"2023-10-18T17:58:23.136521+0000","last_became_active":"2023-10-18T17:58:23.133302+0000","last_became_peered":"2023-10-18T17:58:23.133302+0000","last_unstale":"2023-10-18T17:58:23.136521+0000","last_undegraded":"2023-10-18T17:58:23.136521+0000","last_fullsized":"2023-10-18T17:58:23.136521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104626+0000","last_change":"2023-10-18T17:58:17.792368+0000","last_active":"2023-10-18T17:58:23.104626+0000","last_peered":"2023-10-18T17:58:23.104626+0000","last_clean":"2023-10-18T17:58:23.104626+0000","last_became_active":"2023-10-18T17:58:17.791751+0000","last_became_peered":"2023-10-18T17:58:17.791751+0000","last_unstale":"2023-10-18T17:58:23.104626+0000","last_undegraded":"2023-10-18T17:58:23.104626+0000","last_fullsized":"2023-10-18T17:58:23.104626+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110468+0000","last_change":"2023-10-18T17:58:22.675348+0000","last_active":"2023-10-18T17:58:23.110468+0000","last_peered":"2023-10-18T17:58:23.110468+0000","last_clean":"2023-10-18T17:58:23.110468+0000","last_became_active":"2023-10-18T17:58:22.674469+0000","last_became_peered":"2023-10-18T17:58:22.674469+0000","last_unstale":"2023-10-18T17:58:23.110468+0000","last_undegraded":"2023-10-18T17:58:23.110468+0000","last_fullsized":"2023-10-18T17:58:23.110468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128966+0000","last_change":"2023-10-18T17:58:23.121217+0000","last_active":"2023-10-18T17:58:23.128966+0000","last_peered":"2023-10-18T17:58:23.128966+0000","last_clean":"2023-10-18T17:58:23.128966+0000","last_became_active":"2023-10-18T17:58:23.120691+0000","last_became_peered":"2023-10-18T17:58:23.120691+0000","last_unstale":"2023-10-18T17:58:23.128966+0000","last_undegraded":"2023-10-18T17:58:23.128966+0000","last_fullsized":"2023-10-18T17:58:23.128966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120107+0000","last_change":"2023-10-18T17:58:17.605810+0000","last_active":"2023-10-18T17:58:23.120107+0000","last_peered":"2023-10-18T17:58:23.120107+0000","last_clean":"2023-10-18T17:58:23.120107+0000","last_became_active":"2023-10-18T17:58:17.605289+0000","last_became_peered":"2023-10-18T17:58:17.605289+0000","last_unstale":"2023-10-18T17:58:23.120107+0000","last_undegraded":"2023-10-18T17:58:23.120107+0000","last_fullsized":"2023-10-18T17:58:23.120107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118076+0000","last_change":"2023-10-18T17:58:22.675274+0000","last_active":"2023-10-18T17:58:23.118076+0000","last_peered":"2023-10-18T17:58:23.118076+0000","last_clean":"2023-10-18T17:58:23.118076+0000","last_became_active":"2023-10-18T17:58:22.674965+0000","last_became_peered":"2023-10-18T17:58:22.674965+0000","last_unstale":"2023-10-18T17:58:23.118076+0000","last_undegraded":"2023-10-18T17:58:23.118076+0000","last_fullsized":"2023-10-18T17:58:23.118076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129425+0000","last_change":"2023-10-18T17:58:23.125547+0000","last_active":"2023-10-18T17:58:23.129425+0000","last_peered":"2023-10-18T17:58:23.129425+0000","last_clean":"2023-10-18T17:58:23.129425+0000","last_became_active":"2023-10-18T17:58:23.125259+0000","last_became_peered":"2023-10-18T17:58:23.125259+0000","last_unstale":"2023-10-18T17:58:23.129425+0000","last_undegraded":"2023-10-18T17:58:23.129425+0000","last_fullsized":"2023-10-18T17:58:23.129425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.106468+0000","last_change":"2023-10-18T17:58:18.326048+0000","last_active":"2023-10-18T17:58:23.106468+0000","last_peered":"2023-10-18T17:58:23.106468+0000","last_clean":"2023-10-18T17:58:23.106468+0000","last_became_active":"2023-10-18T17:58:18.325141+0000","last_became_peered":"2023-10-18T17:58:18.325141+0000","last_unstale":"2023-10-18T17:58:23.106468+0000","last_undegraded":"2023-10-18T17:58:23.106468+0000","last_fullsized":"2023-10-18T17:58:23.106468+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114331+0000","last_change":"2023-10-18T17:58:22.684816+0000","last_active":"2023-10-18T17:58:23.114331+0000","last_peered":"2023-10-18T17:58:23.114331+0000","last_clean":"2023-10-18T17:58:23.114331+0000","last_became_active":"2023-10-18T17:58:22.682458+0000","last_became_peered":"2023-10-18T17:58:22.682458+0000","last_unstale":"2023-10-18T17:58:23.114331+0000","last_undegraded":"2023-10-18T17:58:23.114331+0000","last_fullsized":"2023-10-18T17:58:23.114331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134601+0000","last_change":"2023-10-18T17:58:23.131050+0000","last_active":"2023-10-18T17:58:23.134601+0000","last_peered":"2023-10-18T17:58:23.134601+0000","last_clean":"2023-10-18T17:58:23.134601+0000","last_became_active":"2023-10-18T17:58:23.130768+0000","last_became_peered":"2023-10-18T17:58:23.130768+0000","last_unstale":"2023-10-18T17:58:23.134601+0000","last_undegraded":"2023-10-18T17:58:23.134601+0000","last_fullsized":"2023-10-18T17:58:23.134601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110585+0000","last_change":"2023-10-18T17:58:22.698814+0000","last_active":"2023-10-18T17:58:23.110585+0000","last_peered":"2023-10-18T17:58:23.110585+0000","last_clean":"2023-10-18T17:58:23.110585+0000","last_became_active":"2023-10-18T17:58:22.698525+0000","last_became_peered":"2023-10-18T17:58:22.698525+0000","last_unstale":"2023-10-18T17:58:23.110585+0000","last_undegraded":"2023-10-18T17:58:23.110585+0000","last_fullsized":"2023-10-18T17:58:23.110585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117324+0000","last_change":"2023-10-18T17:58:18.322815+0000","last_active":"2023-10-18T17:58:23.117324+0000","last_peered":"2023-10-18T17:58:23.117324+0000","last_clean":"2023-10-18T17:58:23.117324+0000","last_became_active":"2023-10-18T17:58:18.322057+0000","last_became_peered":"2023-10-18T17:58:18.322057+0000","last_unstale":"2023-10-18T17:58:23.117324+0000","last_undegraded":"2023-10-18T17:58:23.117324+0000","last_fullsized":"2023-10-18T17:58:23.117324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.130103+0000","last_change":"2023-10-18T17:58:23.125081+0000","last_active":"2023-10-18T17:58:23.130103+0000","last_peered":"2023-10-18T17:58:23.130103+0000","last_clean":"2023-10-18T17:58:23.130103+0000","last_became_active":"2023-10-18T17:58:23.124833+0000","last_became_peered":"2023-10-18T17:58:23.124833+0000","last_unstale":"2023-10-18T17:58:23.130103+0000","last_undegraded":"2023-10-18T17:58:23.130103+0000","last_fullsized":"2023-10-18T17:58:23.130103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104554+0000","last_change":"2023-10-18T17:58:22.680737+0000","last_active":"2023-10-18T17:58:23.104554+0000","last_peered":"2023-10-18T17:58:23.104554+0000","last_clean":"2023-10-18T17:58:23.104554+0000","last_became_active":"2023-10-18T17:58:22.676981+0000","last_became_peered":"2023-10-18T17:58:22.676981+0000","last_unstale":"2023-10-18T17:58:23.104554+0000","last_undegraded":"2023-10-18T17:58:23.104554+0000","last_fullsized":"2023-10-18T17:58:23.104554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.108047+0000","last_change":"2023-10-18T17:58:17.608107+0000","last_active":"2023-10-18T17:58:23.108047+0000","last_peered":"2023-10-18T17:58:23.108047+0000","last_clean":"2023-10-18T17:58:23.108047+0000","last_became_active":"2023-10-18T17:58:17.607603+0000","last_became_peered":"2023-10-18T17:58:17.607603+0000","last_unstale":"2023-10-18T17:58:23.108047+0000","last_undegraded":"2023-10-18T17:58:23.108047+0000","last_fullsized":"2023-10-18T17:58:23.108047+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129444+0000","last_change":"2023-10-18T17:58:23.125950+0000","last_active":"2023-10-18T17:58:23.129444+0000","last_peered":"2023-10-18T17:58:23.129444+0000","last_clean":"2023-10-18T17:58:23.129444+0000","last_became_active":"2023-10-18T17:58:23.125172+0000","last_became_peered":"2023-10-18T17:58:23.125172+0000","last_unstale":"2023-10-18T17:58:23.129444+0000","last_undegraded":"2023-10-18T17:58:23.129444+0000","last_fullsized":"2023-10-18T17:58:23.129444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117966+0000","last_change":"2023-10-18T17:58:22.683261+0000","last_active":"2023-10-18T17:58:23.117966+0000","last_peered":"2023-10-18T17:58:23.117966+0000","last_clean":"2023-10-18T17:58:23.117966+0000","last_became_active":"2023-10-18T17:58:22.682927+0000","last_became_peered":"2023-10-18T17:58:22.682927+0000","last_unstale":"2023-10-18T17:58:23.117966+0000","last_undegraded":"2023-10-18T17:58:23.117966+0000","last_fullsized":"2023-10-18T17:58:23.117966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114484+0000","last_change":"2023-10-18T17:58:17.607313+0000","last_active":"2023-10-18T17:58:23.114484+0000","last_peered":"2023-10-18T17:58:23.114484+0000","last_clean":"2023-10-18T17:58:23.114484+0000","last_became_active":"2023-10-18T17:58:17.606933+0000","last_became_peered":"2023-10-18T17:58:17.606933+0000","last_unstale":"2023-10-18T17:58:23.114484+0000","last_undegraded":"2023-10-18T17:58:23.114484+0000","last_fullsized":"2023-10-18T17:58:23.114484+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118875+0000","last_change":"2023-10-18T17:58:23.116455+0000","last_active":"2023-10-18T17:58:23.118875+0000","last_peered":"2023-10-18T17:58:23.118875+0000","last_clean":"2023-10-18T17:58:23.118875+0000","last_became_active":"2023-10-18T17:58:23.116050+0000","last_became_peered":"2023-10-18T17:58:23.116050+0000","last_unstale":"2023-10-18T17:58:23.118875+0000","last_undegraded":"2023-10-18T17:58:23.118875+0000","last_fullsized":"2023-10-18T17:58:23.118875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110614+0000","last_change":"2023-10-18T17:58:22.689750+0000","last_active":"2023-10-18T17:58:23.110614+0000","last_peered":"2023-10-18T17:58:23.110614+0000","last_clean":"2023-10-18T17:58:23.110614+0000","last_became_active":"2023-10-18T17:58:22.689474+0000","last_became_peered":"2023-10-18T17:58:22.689474+0000","last_unstale":"2023-10-18T17:58:23.110614+0000","last_undegraded":"2023-10-18T17:58:23.110614+0000","last_fullsized":"2023-10-18T17:58:23.110614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112122+0000","last_change":"2023-10-18T17:58:17.646977+0000","last_active":"2023-10-18T17:58:23.112122+0000","last_peered":"2023-10-18T17:58:23.112122+0000","last_clean":"2023-10-18T17:58:23.112122+0000","last_became_active":"2023-10-18T17:58:17.646568+0000","last_became_peered":"2023-10-18T17:58:17.646568+0000","last_unstale":"2023-10-18T17:58:23.112122+0000","last_undegraded":"2023-10-18T17:58:23.112122+0000","last_fullsized":"2023-10-18T17:58:23.112122+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.101935+0000","last_change":"2023-10-18T17:58:15.581426+0000","last_active":"2023-10-18T17:58:23.101935+0000","last_peered":"2023-10-18T17:58:23.101935+0000","last_clean":"2023-10-18T17:58:23.101935+0000","last_became_active":"2023-10-18T17:58:15.580900+0000","last_became_peered":"2023-10-18T17:58:15.580900+0000","last_unstale":"2023-10-18T17:58:23.101935+0000","last_undegraded":"2023-10-18T17:58:23.101935+0000","last_fullsized":"2023-10-18T17:58:23.101935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:14.550597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:14.550597+0000","last_clean_scrub_stamp":"2023-10-18T17:58:14.550597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135976+0000","last_change":"2023-10-18T17:58:23.130365+0000","last_active":"2023-10-18T17:58:23.135976+0000","last_peered":"2023-10-18T17:58:23.135976+0000","last_clean":"2023-10-18T17:58:23.135976+0000","last_became_active":"2023-10-18T17:58:23.128150+0000","last_became_peered":"2023-10-18T17:58:23.128150+0000","last_unstale":"2023-10-18T17:58:23.135976+0000","last_undegraded":"2023-10-18T17:58:23.135976+0000","last_fullsized":"2023-10-18T17:58:23.135976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110186+0000","last_change":"2023-10-18T17:58:22.675737+0000","last_active":"2023-10-18T17:58:23.110186+0000","last_peered":"2023-10-18T17:58:23.110186+0000","last_clean":"2023-10-18T17:58:23.110186+0000","last_became_active":"2023-10-18T17:58:22.675306+0000","last_became_peered":"2023-10-18T17:58:22.675306+0000","last_unstale":"2023-10-18T17:58:23.110186+0000","last_undegraded":"2023-10-18T17:58:23.110186+0000","last_fullsized":"2023-10-18T17:58:23.110186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119262+0000","last_change":"2023-10-18T17:58:23.115745+0000","last_active":"2023-10-18T17:58:23.119262+0000","last_peered":"2023-10-18T17:58:23.119262+0000","last_clean":"2023-10-18T17:58:23.119262+0000","last_became_active":"2023-10-18T17:58:23.115454+0000","last_became_peered":"2023-10-18T17:58:23.115454+0000","last_unstale":"2023-10-18T17:58:23.119262+0000","last_undegraded":"2023-10-18T17:58:23.119262+0000","last_fullsized":"2023-10-18T17:58:23.119262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.113848+0000","last_change":"2023-10-18T17:58:22.690810+0000","last_active":"2023-10-18T17:58:23.113848+0000","last_peered":"2023-10-18T17:58:23.113848+0000","last_clean":"2023-10-18T17:58:23.113848+0000","last_became_active":"2023-10-18T17:58:22.690506+0000","last_became_peered":"2023-10-18T17:58:22.690506+0000","last_unstale":"2023-10-18T17:58:23.113848+0000","last_undegraded":"2023-10-18T17:58:23.113848+0000","last_fullsized":"2023-10-18T17:58:23.113848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.133051+0000","last_change":"2023-10-18T17:58:23.126643+0000","last_active":"2023-10-18T17:58:23.133051+0000","last_peered":"2023-10-18T17:58:23.133051+0000","last_clean":"2023-10-18T17:58:23.133051+0000","last_became_active":"2023-10-18T17:58:23.126371+0000","last_became_peered":"2023-10-18T17:58:23.126371+0000","last_unstale":"2023-10-18T17:58:23.133051+0000","last_undegraded":"2023-10-18T17:58:23.133051+0000","last_fullsized":"2023-10-18T17:58:23.133051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117250+0000","last_change":"2023-10-18T17:58:22.690893+0000","last_active":"2023-10-18T17:58:23.117250+0000","last_peered":"2023-10-18T17:58:23.117250+0000","last_clean":"2023-10-18T17:58:23.117250+0000","last_became_active":"2023-10-18T17:58:22.689078+0000","last_became_peered":"2023-10-18T17:58:22.689078+0000","last_unstale":"2023-10-18T17:58:23.117250+0000","last_undegraded":"2023-10-18T17:58:23.117250+0000","last_fullsized":"2023-10-18T17:58:23.117250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135993+0000","last_change":"2023-10-18T17:58:23.130578+0000","last_active":"2023-10-18T17:58:23.135993+0000","last_peered":"2023-10-18T17:58:23.135993+0000","last_clean":"2023-10-18T17:58:23.135993+0000","last_became_active":"2023-10-18T17:58:23.130103+0000","last_became_peered":"2023-10-18T17:58:23.130103+0000","last_unstale":"2023-10-18T17:58:23.135993+0000","last_undegraded":"2023-10-18T17:58:23.135993+0000","last_fullsized":"2023-10-18T17:58:23.135993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.108236+0000","last_change":"2023-10-18T17:58:22.682353+0000","last_active":"2023-10-18T17:58:23.108236+0000","last_peered":"2023-10-18T17:58:23.108236+0000","last_clean":"2023-10-18T17:58:23.108236+0000","last_became_active":"2023-10-18T17:58:22.682040+0000","last_became_peered":"2023-10-18T17:58:22.682040+0000","last_unstale":"2023-10-18T17:58:23.108236+0000","last_undegraded":"2023-10-18T17:58:23.108236+0000","last_fullsized":"2023-10-18T17:58:23.108236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127708+0000","last_change":"2023-10-18T17:58:23.120218+0000","last_active":"2023-10-18T17:58:23.127708+0000","last_peered":"2023-10-18T17:58:23.127708+0000","last_clean":"2023-10-18T17:58:23.127708+0000","last_became_active":"2023-10-18T17:58:23.119968+0000","last_became_peered":"2023-10-18T17:58:23.119968+0000","last_unstale":"2023-10-18T17:58:23.127708+0000","last_undegraded":"2023-10-18T17:58:23.127708+0000","last_fullsized":"2023-10-18T17:58:23.127708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110949+0000","last_change":"2023-10-18T17:58:22.679610+0000","last_active":"2023-10-18T17:58:23.110949+0000","last_peered":"2023-10-18T17:58:23.110949+0000","last_clean":"2023-10-18T17:58:23.110949+0000","last_became_active":"2023-10-18T17:58:22.678880+0000","last_became_peered":"2023-10-18T17:58:22.678880+0000","last_unstale":"2023-10-18T17:58:23.110949+0000","last_undegraded":"2023-10-18T17:58:23.110949+0000","last_fullsized":"2023-10-18T17:58:23.110949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120763+0000","last_change":"2023-10-18T17:58:23.118035+0000","last_active":"2023-10-18T17:58:23.120763+0000","last_peered":"2023-10-18T17:58:23.120763+0000","last_clean":"2023-10-18T17:58:23.120763+0000","last_became_active":"2023-10-18T17:58:23.117448+0000","last_became_peered":"2023-10-18T17:58:23.117448+0000","last_unstale":"2023-10-18T17:58:23.120763+0000","last_undegraded":"2023-10-18T17:58:23.120763+0000","last_fullsized":"2023-10-18T17:58:23.120763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119936+0000","last_change":"2023-10-18T17:58:22.688810+0000","last_active":"2023-10-18T17:58:23.119936+0000","last_peered":"2023-10-18T17:58:23.119936+0000","last_clean":"2023-10-18T17:58:23.119936+0000","last_became_active":"2023-10-18T17:58:22.688224+0000","last_became_peered":"2023-10-18T17:58:22.688224+0000","last_unstale":"2023-10-18T17:58:23.119936+0000","last_undegraded":"2023-10-18T17:58:23.119936+0000","last_fullsized":"2023-10-18T17:58:23.119936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136556+0000","last_change":"2023-10-18T17:58:23.133896+0000","last_active":"2023-10-18T17:58:23.136556+0000","last_peered":"2023-10-18T17:58:23.136556+0000","last_clean":"2023-10-18T17:58:23.136556+0000","last_became_active":"2023-10-18T17:58:23.133588+0000","last_became_peered":"2023-10-18T17:58:23.133588+0000","last_unstale":"2023-10-18T17:58:23.136556+0000","last_undegraded":"2023-10-18T17:58:23.136556+0000","last_fullsized":"2023-10-18T17:58:23.136556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116851+0000","last_change":"2023-10-18T17:58:22.696491+0000","last_active":"2023-10-18T17:58:23.116851+0000","last_peered":"2023-10-18T17:58:23.116851+0000","last_clean":"2023-10-18T17:58:23.116851+0000","last_became_active":"2023-10-18T17:58:22.696171+0000","last_became_peered":"2023-10-18T17:58:22.696171+0000","last_unstale":"2023-10-18T17:58:23.116851+0000","last_undegraded":"2023-10-18T17:58:23.116851+0000","last_fullsized":"2023-10-18T17:58:23.116851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125405+0000","last_change":"2023-10-18T17:58:23.116692+0000","last_active":"2023-10-18T17:58:23.125405+0000","last_peered":"2023-10-18T17:58:23.125405+0000","last_clean":"2023-10-18T17:58:23.125405+0000","last_became_active":"2023-10-18T17:58:23.116431+0000","last_became_peered":"2023-10-18T17:58:23.116431+0000","last_unstale":"2023-10-18T17:58:23.125405+0000","last_undegraded":"2023-10-18T17:58:23.125405+0000","last_fullsized":"2023-10-18T17:58:23.125405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111957+0000","last_change":"2023-10-18T17:58:22.694825+0000","last_active":"2023-10-18T17:58:23.111957+0000","last_peered":"2023-10-18T17:58:23.111957+0000","last_clean":"2023-10-18T17:58:23.111957+0000","last_became_active":"2023-10-18T17:58:22.694563+0000","last_became_peered":"2023-10-18T17:58:22.694563+0000","last_unstale":"2023-10-18T17:58:23.111957+0000","last_undegraded":"2023-10-18T17:58:23.111957+0000","last_fullsized":"2023-10-18T17:58:23.111957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126937+0000","last_change":"2023-10-18T17:58:23.120769+0000","last_active":"2023-10-18T17:58:23.126937+0000","last_peered":"2023-10-18T17:58:23.126937+0000","last_clean":"2023-10-18T17:58:23.126937+0000","last_became_active":"2023-10-18T17:58:23.120464+0000","last_became_peered":"2023-10-18T17:58:23.120464+0000","last_unstale":"2023-10-18T17:58:23.126937+0000","last_undegraded":"2023-10-18T17:58:23.126937+0000","last_fullsized":"2023-10-18T17:58:23.126937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.101806+0000","last_change":"2023-10-18T17:58:22.668326+0000","last_active":"2023-10-18T17:58:23.101806+0000","last_peered":"2023-10-18T17:58:23.101806+0000","last_clean":"2023-10-18T17:58:23.101806+0000","last_became_active":"2023-10-18T17:58:22.668059+0000","last_became_peered":"2023-10-18T17:58:22.668059+0000","last_unstale":"2023-10-18T17:58:23.101806+0000","last_undegraded":"2023-10-18T17:58:23.101806+0000","last_fullsized":"2023-10-18T17:58:23.101806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.124121+0000","last_change":"2023-10-18T17:58:23.120596+0000","last_active":"2023-10-18T17:58:23.124121+0000","last_peered":"2023-10-18T17:58:23.124121+0000","last_clean":"2023-10-18T17:58:23.124121+0000","last_became_active":"2023-10-18T17:58:23.120309+0000","last_became_peered":"2023-10-18T17:58:23.120309+0000","last_unstale":"2023-10-18T17:58:23.124121+0000","last_undegraded":"2023-10-18T17:58:23.124121+0000","last_fullsized":"2023-10-18T17:58:23.124121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.113755+0000","last_change":"2023-10-18T17:58:22.700021+0000","last_active":"2023-10-18T17:58:23.113755+0000","last_peered":"2023-10-18T17:58:23.113755+0000","last_clean":"2023-10-18T17:58:23.113755+0000","last_became_active":"2023-10-18T17:58:22.699570+0000","last_became_peered":"2023-10-18T17:58:22.699570+0000","last_unstale":"2023-10-18T17:58:23.113755+0000","last_undegraded":"2023-10-18T17:58:23.113755+0000","last_fullsized":"2023-10-18T17:58:23.113755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121251+0000","last_change":"2023-10-18T17:58:23.118771+0000","last_active":"2023-10-18T17:58:23.121251+0000","last_peered":"2023-10-18T17:58:23.121251+0000","last_clean":"2023-10-18T17:58:23.121251+0000","last_became_active":"2023-10-18T17:58:23.118493+0000","last_became_peered":"2023-10-18T17:58:23.118493+0000","last_unstale":"2023-10-18T17:58:23.121251+0000","last_undegraded":"2023-10-18T17:58:23.121251+0000","last_fullsized":"2023-10-18T17:58:23.121251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117844+0000","last_change":"2023-10-18T17:58:22.694576+0000","last_active":"2023-10-18T17:58:23.117844+0000","last_peered":"2023-10-18T17:58:23.117844+0000","last_clean":"2023-10-18T17:58:23.117844+0000","last_became_active":"2023-10-18T17:58:22.694240+0000","last_became_peered":"2023-10-18T17:58:22.694240+0000","last_unstale":"2023-10-18T17:58:23.117844+0000","last_undegraded":"2023-10-18T17:58:23.117844+0000","last_fullsized":"2023-10-18T17:58:23.117844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134198+0000","last_change":"2023-10-18T17:58:23.126726+0000","last_active":"2023-10-18T17:58:23.134198+0000","last_peered":"2023-10-18T17:58:23.134198+0000","last_clean":"2023-10-18T17:58:23.134198+0000","last_became_active":"2023-10-18T17:58:23.126297+0000","last_became_peered":"2023-10-18T17:58:23.126297+0000","last_unstale":"2023-10-18T17:58:23.134198+0000","last_undegraded":"2023-10-18T17:58:23.134198+0000","last_fullsized":"2023-10-18T17:58:23.134198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112044+0000","last_change":"2023-10-18T17:58:22.690048+0000","last_active":"2023-10-18T17:58:23.112044+0000","last_peered":"2023-10-18T17:58:23.112044+0000","last_clean":"2023-10-18T17:58:23.112044+0000","last_became_active":"2023-10-18T17:58:22.689785+0000","last_became_peered":"2023-10-18T17:58:22.689785+0000","last_unstale":"2023-10-18T17:58:23.112044+0000","last_undegraded":"2023-10-18T17:58:23.112044+0000","last_fullsized":"2023-10-18T17:58:23.112044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134590+0000","last_change":"2023-10-18T17:58:23.129919+0000","last_active":"2023-10-18T17:58:23.134590+0000","last_peered":"2023-10-18T17:58:23.134590+0000","last_clean":"2023-10-18T17:58:23.134590+0000","last_became_active":"2023-10-18T17:58:23.129560+0000","last_became_peered":"2023-10-18T17:58:23.129560+0000","last_unstale":"2023-10-18T17:58:23.134590+0000","last_undegraded":"2023-10-18T17:58:23.134590+0000","last_fullsized":"2023-10-18T17:58:23.134590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118421+0000","last_change":"2023-10-18T17:58:22.678801+0000","last_active":"2023-10-18T17:58:23.118421+0000","last_peered":"2023-10-18T17:58:23.118421+0000","last_clean":"2023-10-18T17:58:23.118421+0000","last_became_active":"2023-10-18T17:58:22.678493+0000","last_became_peered":"2023-10-18T17:58:22.678493+0000","last_unstale":"2023-10-18T17:58:23.118421+0000","last_undegraded":"2023-10-18T17:58:23.118421+0000","last_fullsized":"2023-10-18T17:58:23.118421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129374+0000","last_change":"2023-10-18T17:58:23.124984+0000","last_active":"2023-10-18T17:58:23.129374+0000","last_peered":"2023-10-18T17:58:23.129374+0000","last_clean":"2023-10-18T17:58:23.129374+0000","last_became_active":"2023-10-18T17:58:23.124646+0000","last_became_peered":"2023-10-18T17:58:23.124646+0000","last_unstale":"2023-10-18T17:58:23.129374+0000","last_undegraded":"2023-10-18T17:58:23.129374+0000","last_fullsized":"2023-10-18T17:58:23.129374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118029+0000","last_change":"2023-10-18T17:58:22.677885+0000","last_active":"2023-10-18T17:58:23.118029+0000","last_peered":"2023-10-18T17:58:23.118029+0000","last_clean":"2023-10-18T17:58:23.118029+0000","last_became_active":"2023-10-18T17:58:22.677592+0000","last_became_peered":"2023-10-18T17:58:22.677592+0000","last_unstale":"2023-10-18T17:58:23.118029+0000","last_undegraded":"2023-10-18T17:58:23.118029+0000","last_fullsized":"2023-10-18T17:58:23.118029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117238+0000","last_change":"2023-10-18T17:58:23.115290+0000","last_active":"2023-10-18T17:58:23.117238+0000","last_peered":"2023-10-18T17:58:23.117238+0000","last_clean":"2023-10-18T17:58:23.117238+0000","last_became_active":"2023-10-18T17:58:23.115014+0000","last_became_peered":"2023-10-18T17:58:23.115014+0000","last_unstale":"2023-10-18T17:58:23.117238+0000","last_undegraded":"2023-10-18T17:58:23.117238+0000","last_fullsized":"2023-10-18T17:58:23.117238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110719+0000","last_change":"2023-10-18T17:58:22.695980+0000","last_active":"2023-10-18T17:58:23.110719+0000","last_peered":"2023-10-18T17:58:23.110719+0000","last_clean":"2023-10-18T17:58:23.110719+0000","last_became_active":"2023-10-18T17:58:22.695592+0000","last_became_peered":"2023-10-18T17:58:22.695592+0000","last_unstale":"2023-10-18T17:58:23.110719+0000","last_undegraded":"2023-10-18T17:58:23.110719+0000","last_fullsized":"2023-10-18T17:58:23.110719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127363+0000","last_change":"2023-10-18T17:58:23.124321+0000","last_active":"2023-10-18T17:58:23.127363+0000","last_peered":"2023-10-18T17:58:23.127363+0000","last_clean":"2023-10-18T17:58:23.127363+0000","last_became_active":"2023-10-18T17:58:23.124019+0000","last_became_peered":"2023-10-18T17:58:23.124019+0000","last_unstale":"2023-10-18T17:58:23.127363+0000","last_undegraded":"2023-10-18T17:58:23.127363+0000","last_fullsized":"2023-10-18T17:58:23.127363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111016+0000","last_change":"2023-10-18T17:58:22.692184+0000","last_active":"2023-10-18T17:58:23.111016+0000","last_peered":"2023-10-18T17:58:23.111016+0000","last_clean":"2023-10-18T17:58:23.111016+0000","last_became_active":"2023-10-18T17:58:22.691802+0000","last_became_peered":"2023-10-18T17:58:22.691802+0000","last_unstale":"2023-10-18T17:58:23.111016+0000","last_undegraded":"2023-10-18T17:58:23.111016+0000","last_fullsized":"2023-10-18T17:58:23.111016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126067+0000","last_change":"2023-10-18T17:58:23.116039+0000","last_active":"2023-10-18T17:58:23.126067+0000","last_peered":"2023-10-18T17:58:23.126067+0000","last_clean":"2023-10-18T17:58:23.126067+0000","last_became_active":"2023-10-18T17:58:23.115715+0000","last_became_peered":"2023-10-18T17:58:23.115715+0000","last_unstale":"2023-10-18T17:58:23.126067+0000","last_undegraded":"2023-10-18T17:58:23.126067+0000","last_fullsized":"2023-10-18T17:58:23.126067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111682+0000","last_change":"2023-10-18T17:58:22.687555+0000","last_active":"2023-10-18T17:58:23.111682+0000","last_peered":"2023-10-18T17:58:23.111682+0000","last_clean":"2023-10-18T17:58:23.111682+0000","last_became_active":"2023-10-18T17:58:22.687269+0000","last_became_peered":"2023-10-18T17:58:22.687269+0000","last_unstale":"2023-10-18T17:58:23.111682+0000","last_undegraded":"2023-10-18T17:58:23.111682+0000","last_fullsized":"2023-10-18T17:58:23.111682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127657+0000","last_change":"2023-10-18T17:58:23.120769+0000","last_active":"2023-10-18T17:58:23.127657+0000","last_peered":"2023-10-18T17:58:23.127657+0000","last_clean":"2023-10-18T17:58:23.127657+0000","last_became_active":"2023-10-18T17:58:23.120522+0000","last_became_peered":"2023-10-18T17:58:23.120522+0000","last_unstale":"2023-10-18T17:58:23.127657+0000","last_undegraded":"2023-10-18T17:58:23.127657+0000","last_fullsized":"2023-10-18T17:58:23.127657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104694+0000","last_change":"2023-10-18T17:58:22.689656+0000","last_active":"2023-10-18T17:58:23.104694+0000","last_peered":"2023-10-18T17:58:23.104694+0000","last_clean":"2023-10-18T17:58:23.104694+0000","last_became_active":"2023-10-18T17:58:22.689376+0000","last_became_peered":"2023-10-18T17:58:22.689376+0000","last_unstale":"2023-10-18T17:58:23.104694+0000","last_undegraded":"2023-10-18T17:58:23.104694+0000","last_fullsized":"2023-10-18T17:58:23.104694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126926+0000","last_change":"2023-10-18T17:58:23.120357+0000","last_active":"2023-10-18T17:58:23.126926+0000","last_peered":"2023-10-18T17:58:23.126926+0000","last_clean":"2023-10-18T17:58:23.126926+0000","last_became_active":"2023-10-18T17:58:23.118878+0000","last_became_peered":"2023-10-18T17:58:23.118878+0000","last_unstale":"2023-10-18T17:58:23.126926+0000","last_undegraded":"2023-10-18T17:58:23.126926+0000","last_fullsized":"2023-10-18T17:58:23.126926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128924+0000","last_change":"2023-10-18T17:58:23.126262+0000","last_active":"2023-10-18T17:58:23.128924+0000","last_peered":"2023-10-18T17:58:23.128924+0000","last_clean":"2023-10-18T17:58:23.128924+0000","last_became_active":"2023-10-18T17:58:23.125988+0000","last_became_peered":"2023-10-18T17:58:23.125988+0000","last_unstale":"2023-10-18T17:58:23.128924+0000","last_undegraded":"2023-10-18T17:58:23.128924+0000","last_fullsized":"2023-10-18T17:58:23.128924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104422+0000","last_change":"2023-10-18T17:58:22.680967+0000","last_active":"2023-10-18T17:58:23.104422+0000","last_peered":"2023-10-18T17:58:23.104422+0000","last_clean":"2023-10-18T17:58:23.104422+0000","last_became_active":"2023-10-18T17:58:22.679976+0000","last_became_peered":"2023-10-18T17:58:22.679976+0000","last_unstale":"2023-10-18T17:58:23.104422+0000","last_undegraded":"2023-10-18T17:58:23.104422+0000","last_fullsized":"2023-10-18T17:58:23.104422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135616+0000","last_change":"2023-10-18T17:58:23.131014+0000","last_active":"2023-10-18T17:58:23.135616+0000","last_peered":"2023-10-18T17:58:23.135616+0000","last_clean":"2023-10-18T17:58:23.135616+0000","last_became_active":"2023-10-18T17:58:23.130516+0000","last_became_peered":"2023-10-18T17:58:23.130516+0000","last_unstale":"2023-10-18T17:58:23.135616+0000","last_undegraded":"2023-10-18T17:58:23.135616+0000","last_fullsized":"2023-10-18T17:58:23.135616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.109880+0000","last_change":"2023-10-18T17:58:22.684193+0000","last_active":"2023-10-18T17:58:23.109880+0000","last_peered":"2023-10-18T17:58:23.109880+0000","last_clean":"2023-10-18T17:58:23.109880+0000","last_became_active":"2023-10-18T17:58:22.683668+0000","last_became_peered":"2023-10-18T17:58:22.683668+0000","last_unstale":"2023-10-18T17:58:23.109880+0000","last_undegraded":"2023-10-18T17:58:23.109880+0000","last_fullsized":"2023-10-18T17:58:23.109880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128357+0000","last_change":"2023-10-18T17:58:23.123700+0000","last_active":"2023-10-18T17:58:23.128357+0000","last_peered":"2023-10-18T17:58:23.128357+0000","last_clean":"2023-10-18T17:58:23.128357+0000","last_became_active":"2023-10-18T17:58:23.123224+0000","last_became_peered":"2023-10-18T17:58:23.123224+0000","last_unstale":"2023-10-18T17:58:23.128357+0000","last_undegraded":"2023-10-18T17:58:23.128357+0000","last_fullsized":"2023-10-18T17:58:23.128357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116459+0000","last_change":"2023-10-18T17:58:22.674799+0000","last_active":"2023-10-18T17:58:23.116459+0000","last_peered":"2023-10-18T17:58:23.116459+0000","last_clean":"2023-10-18T17:58:23.116459+0000","last_became_active":"2023-10-18T17:58:22.674479+0000","last_became_peered":"2023-10-18T17:58:22.674479+0000","last_unstale":"2023-10-18T17:58:23.116459+0000","last_undegraded":"2023-10-18T17:58:23.116459+0000","last_fullsized":"2023-10-18T17:58:23.116459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127727+0000","last_change":"2023-10-18T17:58:23.121289+0000","last_active":"2023-10-18T17:58:23.127727+0000","last_peered":"2023-10-18T17:58:23.127727+0000","last_clean":"2023-10-18T17:58:23.127727+0000","last_became_active":"2023-10-18T17:58:23.120024+0000","last_became_peered":"2023-10-18T17:58:23.120024+0000","last_unstale":"2023-10-18T17:58:23.127727+0000","last_undegraded":"2023-10-18T17:58:23.127727+0000","last_fullsized":"2023-10-18T17:58:23.127727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110283+0000","last_change":"2023-10-18T17:58:22.675422+0000","last_active":"2023-10-18T17:58:23.110283+0000","last_peered":"2023-10-18T17:58:23.110283+0000","last_clean":"2023-10-18T17:58:23.110283+0000","last_became_active":"2023-10-18T17:58:22.675074+0000","last_became_peered":"2023-10-18T17:58:22.675074+0000","last_unstale":"2023-10-18T17:58:23.110283+0000","last_undegraded":"2023-10-18T17:58:23.110283+0000","last_fullsized":"2023-10-18T17:58:23.110283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.131984+0000","last_change":"2023-10-18T17:58:23.125653+0000","last_active":"2023-10-18T17:58:23.131984+0000","last_peered":"2023-10-18T17:58:23.131984+0000","last_clean":"2023-10-18T17:58:23.131984+0000","last_became_active":"2023-10-18T17:58:23.124682+0000","last_became_peered":"2023-10-18T17:58:23.124682+0000","last_unstale":"2023-10-18T17:58:23.131984+0000","last_undegraded":"2023-10-18T17:58:23.131984+0000","last_fullsized":"2023-10-18T17:58:23.131984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115530+0000","last_change":"2023-10-18T17:58:22.673247+0000","last_active":"2023-10-18T17:58:23.115530+0000","last_peered":"2023-10-18T17:58:23.115530+0000","last_clean":"2023-10-18T17:58:23.115530+0000","last_became_active":"2023-10-18T17:58:22.672903+0000","last_became_peered":"2023-10-18T17:58:22.672903+0000","last_unstale":"2023-10-18T17:58:23.115530+0000","last_undegraded":"2023-10-18T17:58:23.115530+0000","last_fullsized":"2023-10-18T17:58:23.115530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129965+0000","last_change":"2023-10-18T17:58:23.125035+0000","last_active":"2023-10-18T17:58:23.129965+0000","last_peered":"2023-10-18T17:58:23.129965+0000","last_clean":"2023-10-18T17:58:23.129965+0000","last_became_active":"2023-10-18T17:58:23.124761+0000","last_became_peered":"2023-10-18T17:58:23.124761+0000","last_unstale":"2023-10-18T17:58:23.129965+0000","last_undegraded":"2023-10-18T17:58:23.129965+0000","last_fullsized":"2023-10-18T17:58:23.129965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.106275+0000","last_change":"2023-10-18T17:58:22.672712+0000","last_active":"2023-10-18T17:58:23.106275+0000","last_peered":"2023-10-18T17:58:23.106275+0000","last_clean":"2023-10-18T17:58:23.106275+0000","last_became_active":"2023-10-18T17:58:22.672368+0000","last_became_peered":"2023-10-18T17:58:22.672368+0000","last_unstale":"2023-10-18T17:58:23.106275+0000","last_undegraded":"2023-10-18T17:58:23.106275+0000","last_fullsized":"2023-10-18T17:58:23.106275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135798+0000","last_change":"2023-10-18T17:58:23.127848+0000","last_active":"2023-10-18T17:58:23.135798+0000","last_peered":"2023-10-18T17:58:23.135798+0000","last_clean":"2023-10-18T17:58:23.135798+0000","last_became_active":"2023-10-18T17:58:23.125611+0000","last_became_peered":"2023-10-18T17:58:23.125611+0000","last_unstale":"2023-10-18T17:58:23.135798+0000","last_undegraded":"2023-10-18T17:58:23.135798+0000","last_fullsized":"2023-10-18T17:58:23.135798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104555+0000","last_change":"2023-10-18T17:58:22.675355+0000","last_active":"2023-10-18T17:58:23.104555+0000","last_peered":"2023-10-18T17:58:23.104555+0000","last_clean":"2023-10-18T17:58:23.104555+0000","last_became_active":"2023-10-18T17:58:22.675050+0000","last_became_peered":"2023-10-18T17:58:22.675050+0000","last_unstale":"2023-10-18T17:58:23.104555+0000","last_undegraded":"2023-10-18T17:58:23.104555+0000","last_fullsized":"2023-10-18T17:58:23.104555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129429+0000","last_change":"2023-10-18T17:58:23.124889+0000","last_active":"2023-10-18T17:58:23.129429+0000","last_peered":"2023-10-18T17:58:23.129429+0000","last_clean":"2023-10-18T17:58:23.129429+0000","last_became_active":"2023-10-18T17:58:23.124619+0000","last_became_peered":"2023-10-18T17:58:23.124619+0000","last_unstale":"2023-10-18T17:58:23.129429+0000","last_undegraded":"2023-10-18T17:58:23.129429+0000","last_fullsized":"2023-10-18T17:58:23.129429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111694+0000","last_change":"2023-10-18T17:58:22.692437+0000","last_active":"2023-10-18T17:58:23.111694+0000","last_peered":"2023-10-18T17:58:23.111694+0000","last_clean":"2023-10-18T17:58:23.111694+0000","last_became_active":"2023-10-18T17:58:22.692043+0000","last_became_peered":"2023-10-18T17:58:22.692043+0000","last_unstale":"2023-10-18T17:58:23.111694+0000","last_undegraded":"2023-10-18T17:58:23.111694+0000","last_fullsized":"2023-10-18T17:58:23.111694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T17:58:36.293 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T17:58:36.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd0b379700 1 Processor -- start 2023-10-18T17:58:36.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd0b379700 1 -- start start 2023-10-18T17:58:36.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd0b379700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdd041437e0 0x7fdd040bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd0b379700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fdd0414d750 0x7fdd040bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd0b379700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd041501e0 con 0x7fdd04143400 2023-10-18T17:58:36.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd0b379700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd041504c0 con 0x7fdd041437e0 2023-10-18T17:58:36.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd0b379700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd041507a0 con 0x7fdd0414d750 2023-10-18T17:58:36.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd09b76700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdd041437e0 0x7fdd040bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd09b76700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fdd041437e0 0x7fdd040bf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36552/0 (socket says 172.21.15.72:36552) 2023-10-18T17:58:36.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd09b76700 1 -- 172.21.15.72:0/588605699 learned_addr learned my addr 172.21.15.72:0/588605699 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:36.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd0ab78700 1 --2- 172.21.15.72:0/588605699 >> v2:172.21.15.179:3300/0 conn(0x7fdd0414d750 0x7fdd040bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.486+0000 7fdd09b76700 1 -- 172.21.15.72:0/588605699 >> v2:172.21.15.179:3300/0 conn(0x7fdd0414d750 msgr2=0x7fdd040bf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:36.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd09b76700 1 --2- 172.21.15.72:0/588605699 >> v2:172.21.15.179:3300/0 conn(0x7fdd0414d750 0x7fdd040bf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd09b76700 1 -- 172.21.15.72:0/588605699 >> v1:172.21.15.72:6789/0 conn(0x7fdd04143400 legacy=0x7fdd040bd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:36.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd09b76700 1 -- 172.21.15.72:0/588605699 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd04150a80 con 0x7fdd041437e0 2023-10-18T17:58:36.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd0ab78700 1 --2- 172.21.15.72:0/588605699 >> v2:172.21.15.179:3300/0 conn(0x7fdd0414d750 0x7fdd040bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-18T17:58:36.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd09b76700 1 --2- 172.21.15.72:0/588605699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd041437e0 0x7fdd040bf1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fdd040be800 tx=0x7fdcf40090f0).ready entity=mon.0 client_cookie=e42319966840e381 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:36.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd09375700 1 -- 172.21.15.72:0/588605699 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400b630 con 0x7fdd041437e0 2023-10-18T17:58:36.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd09375700 1 -- 172.21.15.72:0/588605699 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf400b790 con 0x7fdd041437e0 2023-10-18T17:58:36.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd09375700 1 -- 172.21.15.72:0/588605699 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf400bb10 con 0x7fdd041437e0 2023-10-18T17:58:36.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd0b379700 1 -- 172.21.15.72:0/588605699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd041437e0 msgr2=0x7fdd040bf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd0b379700 1 --2- 172.21.15.72:0/588605699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd041437e0 0x7fdd040bf1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fdd040be800 tx=0x7fdcf40090f0).stop 2023-10-18T17:58:36.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd0b379700 1 -- 172.21.15.72:0/588605699 shutdown_connections 2023-10-18T17:58:36.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd0b379700 1 --2- 172.21.15.72:0/588605699 >> v2:172.21.15.179:3300/0 conn(0x7fdd0414d750 0x7fdd040bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd0b379700 1 --2- 172.21.15.72:0/588605699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd041437e0 0x7fdd040bf1a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd0b379700 1 -- 172.21.15.72:0/588605699 >> 172.21.15.72:0/588605699 conn(0x7fdd0405ddc0 msgr2=0x7fdd0405e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:36.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd0b379700 1 -- 172.21.15.72:0/588605699 shutdown_connections 2023-10-18T17:58:36.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.490+0000 7fdd0b379700 1 -- 172.21.15.72:0/588605699 wait complete. 2023-10-18T17:58:36.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0b379700 1 Processor -- start 2023-10-18T17:58:36.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0b379700 1 -- start start 2023-10-18T17:58:36.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0b379700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd04143400 0x7fdd0414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0b379700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd041437e0 0x7fdd0413d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0b379700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd0414d750 0x7fdd0414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0b379700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04160040 con 0x7fdd04143400 2023-10-18T17:58:36.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0b379700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04160320 con 0x7fdd041437e0 2023-10-18T17:58:36.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0b379700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04160600 con 0x7fdd0414d750 2023-10-18T17:58:36.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0a377700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd04143400 0x7fdd0414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0a377700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd04143400 0x7fdd0414f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36554/0 (socket says 172.21.15.72:36554) 2023-10-18T17:58:36.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0a377700 1 -- 172.21.15.72:0/2706373223 learned_addr learned my addr 172.21.15.72:0/2706373223 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:36.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd09b76700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd041437e0 0x7fdd0413d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.494+0000 7fdd0ab78700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd0414d750 0x7fdd0414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd0a377700 1 -- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd0414d750 msgr2=0x7fdd0414e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd0a377700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd0414d750 0x7fdd0414e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd0a377700 1 -- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd041437e0 msgr2=0x7fdd0413d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd0a377700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd041437e0 0x7fdd0413d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd0a377700 1 -- 172.21.15.72:0/2706373223 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf4009370 con 0x7fdd04143400 2023-10-18T17:58:36.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd09b76700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd041437e0 0x7fdd0413d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:36.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd0a377700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd04143400 0x7fdd0414f7e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fdd0000c440 tx=0x7fdd00007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:36.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdcfa7fc700 1 -- 172.21.15.72:0/2706373223 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000ce80 con 0x7fdd04143400 2023-10-18T17:58:36.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdcfa7fc700 1 -- 172.21.15.72:0/2706373223 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0000f470 con 0x7fdd04143400 2023-10-18T17:58:36.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd041608e0 con 0x7fdd04143400 2023-10-18T17:58:36.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdcfa7fc700 1 -- 172.21.15.72:0/2706373223 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000f830 con 0x7fdd04143400 2023-10-18T17:58:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0414fe90 con 0x7fdd04143400 2023-10-18T17:58:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.498+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd041429c0 con 0x7fdd04143400 2023-10-18T17:58:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.502+0000 7fdcfa7fc700 1 -- 172.21.15.72:0/2706373223 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdd0000fc20 con 0x7fdd04143400 2023-10-18T17:58:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.502+0000 7fdcfa7fc700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdce805a540 0x7fdce805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.502+0000 7fdcfa7fc700 1 -- 172.21.15.72:0/2706373223 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdd00073c20 con 0x7fdd04143400 2023-10-18T17:58:36.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.502+0000 7fdd09b76700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdce805a540 0x7fdce805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.502+0000 7fdd09b76700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdce805a540 0x7fdce805ca00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fdd04143bc0 tx=0x7fdcf4015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:36.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.506+0000 7fdcfa7fc700 1 -- 172.21.15.72:0/2706373223 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fdd00047430 con 0x7fdd04143400 2023-10-18T17:58:36.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.666+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 --> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd0414df00 con 0x7fdce805a540 2023-10-18T17:58:36.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.686+0000 7fdcfa7fc700 1 -- 172.21.15.72:0/2706373223 <== mgr.4102 v2:172.21.15.179:6832/33696 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336091 (secure 0 0 0) 0x7fdd0414df00 con 0x7fdce805a540 2023-10-18T17:58:36.691 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:36.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdce805a540 msgr2=0x7fdce805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdce805a540 0x7fdce805ca00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fdd04143bc0 tx=0x7fdcf4015000).stop 2023-10-18T17:58:36.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd04143400 msgr2=0x7fdd0414f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd04143400 0x7fdd0414f7e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fdd0000c440 tx=0x7fdd00007de0).stop 2023-10-18T17:58:36.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 shutdown_connections 2023-10-18T17:58:36.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdce805a540 0x7fdce805ca00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdd0414d750 0x7fdd0414e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd041437e0 0x7fdd0413d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 --2- 172.21.15.72:0/2706373223 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd04143400 0x7fdd0414f7e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 >> 172.21.15.72:0/2706373223 conn(0x7fdd0405ddc0 msgr2=0x7fdd0414cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:36.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 shutdown_connections 2023-10-18T17:58:36.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.694+0000 7fdd0b379700 1 -- 172.21.15.72:0/2706373223 wait complete. 2023-10-18T17:58:36.700 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-10-18T17:58:36.719 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-18T17:58:36.471391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378528,"kb_used_data":1760,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596192,"statfs":{"total":773094113280,"available":770658500608,"internally_reserved":0,"allocated":1802240,"data_stored":677080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129427+0000","last_change":"2023-10-18T17:58:23.125789+0000","last_active":"2023-10-18T17:58:23.129427+0000","last_peered":"2023-10-18T17:58:23.129427+0000","last_clean":"2023-10-18T17:58:23.129427+0000","last_became_active":"2023-10-18T17:58:23.123952+0000","last_became_peered":"2023-10-18T17:58:23.123952+0000","last_unstale":"2023-10-18T17:58:23.129427+0000","last_undegraded":"2023-10-18T17:58:23.129427+0000","last_fullsized":"2023-10-18T17:58:23.129427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103779+0000","last_change":"2023-10-18T17:58:22.680518+0000","last_active":"2023-10-18T17:58:23.103779+0000","last_peered":"2023-10-18T17:58:23.103779+0000","last_clean":"2023-10-18T17:58:23.103779+0000","last_became_active":"2023-10-18T17:58:22.679123+0000","last_became_peered":"2023-10-18T17:58:22.679123+0000","last_unstale":"2023-10-18T17:58:23.103779+0000","last_undegraded":"2023-10-18T17:58:23.103779+0000","last_fullsized":"2023-10-18T17:58:23.103779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129023+0000","last_change":"2023-10-18T17:58:23.121498+0000","last_active":"2023-10-18T17:58:23.129023+0000","last_peered":"2023-10-18T17:58:23.129023+0000","last_clean":"2023-10-18T17:58:23.129023+0000","last_became_active":"2023-10-18T17:58:23.121129+0000","last_became_peered":"2023-10-18T17:58:23.121129+0000","last_unstale":"2023-10-18T17:58:23.129023+0000","last_undegraded":"2023-10-18T17:58:23.129023+0000","last_fullsized":"2023-10-18T17:58:23.129023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114330+0000","last_change":"2023-10-18T17:58:22.701312+0000","last_active":"2023-10-18T17:58:23.114330+0000","last_peered":"2023-10-18T17:58:23.114330+0000","last_clean":"2023-10-18T17:58:23.114330+0000","last_became_active":"2023-10-18T17:58:22.701013+0000","last_became_peered":"2023-10-18T17:58:22.701013+0000","last_unstale":"2023-10-18T17:58:23.114330+0000","last_undegraded":"2023-10-18T17:58:23.114330+0000","last_fullsized":"2023-10-18T17:58:23.114330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120622+0000","last_change":"2023-10-18T17:58:22.690786+0000","last_active":"2023-10-18T17:58:23.120622+0000","last_peered":"2023-10-18T17:58:23.120622+0000","last_clean":"2023-10-18T17:58:23.120622+0000","last_became_active":"2023-10-18T17:58:22.690530+0000","last_became_peered":"2023-10-18T17:58:22.690530+0000","last_unstale":"2023-10-18T17:58:23.120622+0000","last_undegraded":"2023-10-18T17:58:23.120622+0000","last_fullsized":"2023-10-18T17:58:23.120622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121219+0000","last_change":"2023-10-18T17:58:23.112227+0000","last_active":"2023-10-18T17:58:23.121219+0000","last_peered":"2023-10-18T17:58:23.121219+0000","last_clean":"2023-10-18T17:58:23.121219+0000","last_became_active":"2023-10-18T17:58:23.111236+0000","last_became_peered":"2023-10-18T17:58:23.111236+0000","last_unstale":"2023-10-18T17:58:23.121219+0000","last_undegraded":"2023-10-18T17:58:23.121219+0000","last_fullsized":"2023-10-18T17:58:23.121219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.109161+0000","last_change":"2023-10-18T17:58:22.700039+0000","last_active":"2023-10-18T17:58:23.109161+0000","last_peered":"2023-10-18T17:58:23.109161+0000","last_clean":"2023-10-18T17:58:23.109161+0000","last_became_active":"2023-10-18T17:58:22.699719+0000","last_became_peered":"2023-10-18T17:58:22.699719+0000","last_unstale":"2023-10-18T17:58:23.109161+0000","last_undegraded":"2023-10-18T17:58:23.109161+0000","last_fullsized":"2023-10-18T17:58:23.109161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128700+0000","last_change":"2023-10-18T17:58:23.116723+0000","last_active":"2023-10-18T17:58:23.128700+0000","last_peered":"2023-10-18T17:58:23.128700+0000","last_clean":"2023-10-18T17:58:23.128700+0000","last_became_active":"2023-10-18T17:58:23.116376+0000","last_became_peered":"2023-10-18T17:58:23.116376+0000","last_unstale":"2023-10-18T17:58:23.128700+0000","last_undegraded":"2023-10-18T17:58:23.128700+0000","last_fullsized":"2023-10-18T17:58:23.128700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120037+0000","last_change":"2023-10-18T17:58:22.680018+0000","last_active":"2023-10-18T17:58:23.120037+0000","last_peered":"2023-10-18T17:58:23.120037+0000","last_clean":"2023-10-18T17:58:23.120037+0000","last_became_active":"2023-10-18T17:58:22.679464+0000","last_became_peered":"2023-10-18T17:58:22.679464+0000","last_unstale":"2023-10-18T17:58:23.120037+0000","last_undegraded":"2023-10-18T17:58:23.120037+0000","last_fullsized":"2023-10-18T17:58:23.120037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119206+0000","last_change":"2023-10-18T17:58:23.115758+0000","last_active":"2023-10-18T17:58:23.119206+0000","last_peered":"2023-10-18T17:58:23.119206+0000","last_clean":"2023-10-18T17:58:23.119206+0000","last_became_active":"2023-10-18T17:58:23.115471+0000","last_became_peered":"2023-10-18T17:58:23.115471+0000","last_unstale":"2023-10-18T17:58:23.119206+0000","last_undegraded":"2023-10-18T17:58:23.119206+0000","last_fullsized":"2023-10-18T17:58:23.119206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110821+0000","last_change":"2023-10-18T17:58:22.686370+0000","last_active":"2023-10-18T17:58:23.110821+0000","last_peered":"2023-10-18T17:58:23.110821+0000","last_clean":"2023-10-18T17:58:23.110821+0000","last_became_active":"2023-10-18T17:58:22.686062+0000","last_became_peered":"2023-10-18T17:58:22.686062+0000","last_unstale":"2023-10-18T17:58:23.110821+0000","last_undegraded":"2023-10-18T17:58:23.110821+0000","last_fullsized":"2023-10-18T17:58:23.110821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125366+0000","last_change":"2023-10-18T17:58:23.118319+0000","last_active":"2023-10-18T17:58:23.125366+0000","last_peered":"2023-10-18T17:58:23.125366+0000","last_clean":"2023-10-18T17:58:23.125366+0000","last_became_active":"2023-10-18T17:58:23.118018+0000","last_became_peered":"2023-10-18T17:58:23.118018+0000","last_unstale":"2023-10-18T17:58:23.125366+0000","last_undegraded":"2023-10-18T17:58:23.125366+0000","last_fullsized":"2023-10-18T17:58:23.125366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103546+0000","last_change":"2023-10-18T17:58:22.697114+0000","last_active":"2023-10-18T17:58:23.103546+0000","last_peered":"2023-10-18T17:58:23.103546+0000","last_clean":"2023-10-18T17:58:23.103546+0000","last_became_active":"2023-10-18T17:58:22.696798+0000","last_became_peered":"2023-10-18T17:58:22.696798+0000","last_unstale":"2023-10-18T17:58:23.103546+0000","last_undegraded":"2023-10-18T17:58:23.103546+0000","last_fullsized":"2023-10-18T17:58:23.103546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125558+0000","last_change":"2023-10-18T17:58:23.117398+0000","last_active":"2023-10-18T17:58:23.125558+0000","last_peered":"2023-10-18T17:58:23.125558+0000","last_clean":"2023-10-18T17:58:23.125558+0000","last_became_active":"2023-10-18T17:58:23.117086+0000","last_became_peered":"2023-10-18T17:58:23.117086+0000","last_unstale":"2023-10-18T17:58:23.125558+0000","last_undegraded":"2023-10-18T17:58:23.125558+0000","last_fullsized":"2023-10-18T17:58:23.125558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103780+0000","last_change":"2023-10-18T17:58:22.678915+0000","last_active":"2023-10-18T17:58:23.103780+0000","last_peered":"2023-10-18T17:58:23.103780+0000","last_clean":"2023-10-18T17:58:23.103780+0000","last_became_active":"2023-10-18T17:58:22.678609+0000","last_became_peered":"2023-10-18T17:58:22.678609+0000","last_unstale":"2023-10-18T17:58:23.103780+0000","last_undegraded":"2023-10-18T17:58:23.103780+0000","last_fullsized":"2023-10-18T17:58:23.103780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125651+0000","last_change":"2023-10-18T17:58:23.122008+0000","last_active":"2023-10-18T17:58:23.125651+0000","last_peered":"2023-10-18T17:58:23.125651+0000","last_clean":"2023-10-18T17:58:23.125651+0000","last_became_active":"2023-10-18T17:58:23.121684+0000","last_became_peered":"2023-10-18T17:58:23.121684+0000","last_unstale":"2023-10-18T17:58:23.125651+0000","last_undegraded":"2023-10-18T17:58:23.125651+0000","last_fullsized":"2023-10-18T17:58:23.125651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103948+0000","last_change":"2023-10-18T17:58:22.668404+0000","last_active":"2023-10-18T17:58:23.103948+0000","last_peered":"2023-10-18T17:58:23.103948+0000","last_clean":"2023-10-18T17:58:23.103948+0000","last_became_active":"2023-10-18T17:58:22.668038+0000","last_became_peered":"2023-10-18T17:58:22.668038+0000","last_unstale":"2023-10-18T17:58:23.103948+0000","last_undegraded":"2023-10-18T17:58:23.103948+0000","last_fullsized":"2023-10-18T17:58:23.103948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127151+0000","last_change":"2023-10-18T17:58:23.121199+0000","last_active":"2023-10-18T17:58:23.127151+0000","last_peered":"2023-10-18T17:58:23.127151+0000","last_clean":"2023-10-18T17:58:23.127151+0000","last_became_active":"2023-10-18T17:58:23.120856+0000","last_became_peered":"2023-10-18T17:58:23.120856+0000","last_unstale":"2023-10-18T17:58:23.127151+0000","last_undegraded":"2023-10-18T17:58:23.127151+0000","last_fullsized":"2023-10-18T17:58:23.127151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114412+0000","last_change":"2023-10-18T17:58:22.688492+0000","last_active":"2023-10-18T17:58:23.114412+0000","last_peered":"2023-10-18T17:58:23.114412+0000","last_clean":"2023-10-18T17:58:23.114412+0000","last_became_active":"2023-10-18T17:58:22.688162+0000","last_became_peered":"2023-10-18T17:58:22.688162+0000","last_unstale":"2023-10-18T17:58:23.114412+0000","last_undegraded":"2023-10-18T17:58:23.114412+0000","last_fullsized":"2023-10-18T17:58:23.114412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128660+0000","last_change":"2023-10-18T17:58:23.125037+0000","last_active":"2023-10-18T17:58:23.128660+0000","last_peered":"2023-10-18T17:58:23.128660+0000","last_clean":"2023-10-18T17:58:23.128660+0000","last_became_active":"2023-10-18T17:58:23.124722+0000","last_became_peered":"2023-10-18T17:58:23.124722+0000","last_unstale":"2023-10-18T17:58:23.128660+0000","last_undegraded":"2023-10-18T17:58:23.128660+0000","last_fullsized":"2023-10-18T17:58:23.128660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103799+0000","last_change":"2023-10-18T17:58:22.680359+0000","last_active":"2023-10-18T17:58:23.103799+0000","last_peered":"2023-10-18T17:58:23.103799+0000","last_clean":"2023-10-18T17:58:23.103799+0000","last_became_active":"2023-10-18T17:58:22.679356+0000","last_became_peered":"2023-10-18T17:58:22.679356+0000","last_unstale":"2023-10-18T17:58:23.103799+0000","last_undegraded":"2023-10-18T17:58:23.103799+0000","last_fullsized":"2023-10-18T17:58:23.103799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129449+0000","last_change":"2023-10-18T17:58:23.125443+0000","last_active":"2023-10-18T17:58:23.129449+0000","last_peered":"2023-10-18T17:58:23.129449+0000","last_clean":"2023-10-18T17:58:23.129449+0000","last_became_active":"2023-10-18T17:58:23.124036+0000","last_became_peered":"2023-10-18T17:58:23.124036+0000","last_unstale":"2023-10-18T17:58:23.129449+0000","last_undegraded":"2023-10-18T17:58:23.129449+0000","last_fullsized":"2023-10-18T17:58:23.129449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116673+0000","last_change":"2023-10-18T17:58:22.695695+0000","last_active":"2023-10-18T17:58:23.116673+0000","last_peered":"2023-10-18T17:58:23.116673+0000","last_clean":"2023-10-18T17:58:23.116673+0000","last_became_active":"2023-10-18T17:58:22.694956+0000","last_became_peered":"2023-10-18T17:58:22.694956+0000","last_unstale":"2023-10-18T17:58:23.116673+0000","last_undegraded":"2023-10-18T17:58:23.116673+0000","last_fullsized":"2023-10-18T17:58:23.116673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127389+0000","last_change":"2023-10-18T17:58:23.124170+0000","last_active":"2023-10-18T17:58:23.127389+0000","last_peered":"2023-10-18T17:58:23.127389+0000","last_clean":"2023-10-18T17:58:23.127389+0000","last_became_active":"2023-10-18T17:58:23.123835+0000","last_became_peered":"2023-10-18T17:58:23.123835+0000","last_unstale":"2023-10-18T17:58:23.127389+0000","last_undegraded":"2023-10-18T17:58:23.127389+0000","last_fullsized":"2023-10-18T17:58:23.127389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118363+0000","last_change":"2023-10-18T17:58:22.678817+0000","last_active":"2023-10-18T17:58:23.118363+0000","last_peered":"2023-10-18T17:58:23.118363+0000","last_clean":"2023-10-18T17:58:23.118363+0000","last_became_active":"2023-10-18T17:58:22.678508+0000","last_became_peered":"2023-10-18T17:58:22.678508+0000","last_unstale":"2023-10-18T17:58:23.118363+0000","last_undegraded":"2023-10-18T17:58:23.118363+0000","last_fullsized":"2023-10-18T17:58:23.118363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.133265+0000","last_change":"2023-10-18T17:58:23.129445+0000","last_active":"2023-10-18T17:58:23.133265+0000","last_peered":"2023-10-18T17:58:23.133265+0000","last_clean":"2023-10-18T17:58:23.133265+0000","last_became_active":"2023-10-18T17:58:23.129038+0000","last_became_peered":"2023-10-18T17:58:23.129038+0000","last_unstale":"2023-10-18T17:58:23.133265+0000","last_undegraded":"2023-10-18T17:58:23.133265+0000","last_fullsized":"2023-10-18T17:58:23.133265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117910+0000","last_change":"2023-10-18T17:58:22.684469+0000","last_active":"2023-10-18T17:58:23.117910+0000","last_peered":"2023-10-18T17:58:23.117910+0000","last_clean":"2023-10-18T17:58:23.117910+0000","last_became_active":"2023-10-18T17:58:22.684160+0000","last_became_peered":"2023-10-18T17:58:22.684160+0000","last_unstale":"2023-10-18T17:58:23.117910+0000","last_undegraded":"2023-10-18T17:58:23.117910+0000","last_fullsized":"2023-10-18T17:58:23.117910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125948+0000","last_change":"2023-10-18T17:58:23.120664+0000","last_active":"2023-10-18T17:58:23.125948+0000","last_peered":"2023-10-18T17:58:23.125948+0000","last_clean":"2023-10-18T17:58:23.125948+0000","last_became_active":"2023-10-18T17:58:23.120359+0000","last_became_peered":"2023-10-18T17:58:23.120359+0000","last_unstale":"2023-10-18T17:58:23.125948+0000","last_undegraded":"2023-10-18T17:58:23.125948+0000","last_fullsized":"2023-10-18T17:58:23.125948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.105546+0000","last_change":"2023-10-18T17:58:22.676645+0000","last_active":"2023-10-18T17:58:23.105546+0000","last_peered":"2023-10-18T17:58:23.105546+0000","last_clean":"2023-10-18T17:58:23.105546+0000","last_became_active":"2023-10-18T17:58:22.676295+0000","last_became_peered":"2023-10-18T17:58:22.676295+0000","last_unstale":"2023-10-18T17:58:23.105546+0000","last_undegraded":"2023-10-18T17:58:23.105546+0000","last_fullsized":"2023-10-18T17:58:23.105546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127404+0000","last_change":"2023-10-18T17:58:23.124501+0000","last_active":"2023-10-18T17:58:23.127404+0000","last_peered":"2023-10-18T17:58:23.127404+0000","last_clean":"2023-10-18T17:58:23.127404+0000","last_became_active":"2023-10-18T17:58:23.124186+0000","last_became_peered":"2023-10-18T17:58:23.124186+0000","last_unstale":"2023-10-18T17:58:23.127404+0000","last_undegraded":"2023-10-18T17:58:23.127404+0000","last_fullsized":"2023-10-18T17:58:23.127404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117130+0000","last_change":"2023-10-18T17:58:22.690272+0000","last_active":"2023-10-18T17:58:23.117130+0000","last_peered":"2023-10-18T17:58:23.117130+0000","last_clean":"2023-10-18T17:58:23.117130+0000","last_became_active":"2023-10-18T17:58:22.689949+0000","last_became_peered":"2023-10-18T17:58:22.689949+0000","last_unstale":"2023-10-18T17:58:23.117130+0000","last_undegraded":"2023-10-18T17:58:23.117130+0000","last_fullsized":"2023-10-18T17:58:23.117130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134179+0000","last_change":"2023-10-18T17:58:23.126544+0000","last_active":"2023-10-18T17:58:23.134179+0000","last_peered":"2023-10-18T17:58:23.134179+0000","last_clean":"2023-10-18T17:58:23.134179+0000","last_became_active":"2023-10-18T17:58:23.125987+0000","last_became_peered":"2023-10-18T17:58:23.125987+0000","last_unstale":"2023-10-18T17:58:23.134179+0000","last_undegraded":"2023-10-18T17:58:23.134179+0000","last_fullsized":"2023-10-18T17:58:23.134179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.107900+0000","last_change":"2023-10-18T17:58:22.681429+0000","last_active":"2023-10-18T17:58:23.107900+0000","last_peered":"2023-10-18T17:58:23.107900+0000","last_clean":"2023-10-18T17:58:23.107900+0000","last_became_active":"2023-10-18T17:58:22.681145+0000","last_became_peered":"2023-10-18T17:58:22.681145+0000","last_unstale":"2023-10-18T17:58:23.107900+0000","last_undegraded":"2023-10-18T17:58:23.107900+0000","last_fullsized":"2023-10-18T17:58:23.107900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121294+0000","last_change":"2023-10-18T17:58:23.117504+0000","last_active":"2023-10-18T17:58:23.121294+0000","last_peered":"2023-10-18T17:58:23.121294+0000","last_clean":"2023-10-18T17:58:23.121294+0000","last_became_active":"2023-10-18T17:58:23.117192+0000","last_became_peered":"2023-10-18T17:58:23.117192+0000","last_unstale":"2023-10-18T17:58:23.121294+0000","last_undegraded":"2023-10-18T17:58:23.121294+0000","last_fullsized":"2023-10-18T17:58:23.121294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114324+0000","last_change":"2023-10-18T17:58:22.682889+0000","last_active":"2023-10-18T17:58:23.114324+0000","last_peered":"2023-10-18T17:58:23.114324+0000","last_clean":"2023-10-18T17:58:23.114324+0000","last_became_active":"2023-10-18T17:58:22.682273+0000","last_became_peered":"2023-10-18T17:58:22.682273+0000","last_unstale":"2023-10-18T17:58:23.114324+0000","last_undegraded":"2023-10-18T17:58:23.114324+0000","last_fullsized":"2023-10-18T17:58:23.114324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.132038+0000","last_change":"2023-10-18T17:58:23.126856+0000","last_active":"2023-10-18T17:58:23.132038+0000","last_peered":"2023-10-18T17:58:23.132038+0000","last_clean":"2023-10-18T17:58:23.132038+0000","last_became_active":"2023-10-18T17:58:23.126582+0000","last_became_peered":"2023-10-18T17:58:23.126582+0000","last_unstale":"2023-10-18T17:58:23.132038+0000","last_undegraded":"2023-10-18T17:58:23.132038+0000","last_fullsized":"2023-10-18T17:58:23.132038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104415+0000","last_change":"2023-10-18T17:58:22.680971+0000","last_active":"2023-10-18T17:58:23.104415+0000","last_peered":"2023-10-18T17:58:23.104415+0000","last_clean":"2023-10-18T17:58:23.104415+0000","last_became_active":"2023-10-18T17:58:22.678750+0000","last_became_peered":"2023-10-18T17:58:22.678750+0000","last_unstale":"2023-10-18T17:58:23.104415+0000","last_undegraded":"2023-10-18T17:58:23.104415+0000","last_fullsized":"2023-10-18T17:58:23.104415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135050+0000","last_change":"2023-10-18T17:58:23.130754+0000","last_active":"2023-10-18T17:58:23.135050+0000","last_peered":"2023-10-18T17:58:23.135050+0000","last_clean":"2023-10-18T17:58:23.135050+0000","last_became_active":"2023-10-18T17:58:23.130419+0000","last_became_peered":"2023-10-18T17:58:23.130419+0000","last_unstale":"2023-10-18T17:58:23.135050+0000","last_undegraded":"2023-10-18T17:58:23.135050+0000","last_fullsized":"2023-10-18T17:58:23.135050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104721+0000","last_change":"2023-10-18T17:58:22.683291+0000","last_active":"2023-10-18T17:58:23.104721+0000","last_peered":"2023-10-18T17:58:23.104721+0000","last_clean":"2023-10-18T17:58:23.104721+0000","last_became_active":"2023-10-18T17:58:22.682926+0000","last_became_peered":"2023-10-18T17:58:22.682926+0000","last_unstale":"2023-10-18T17:58:23.104721+0000","last_undegraded":"2023-10-18T17:58:23.104721+0000","last_fullsized":"2023-10-18T17:58:23.104721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128937+0000","last_change":"2023-10-18T17:58:23.126426+0000","last_active":"2023-10-18T17:58:23.128937+0000","last_peered":"2023-10-18T17:58:23.128937+0000","last_clean":"2023-10-18T17:58:23.128937+0000","last_became_active":"2023-10-18T17:58:23.126114+0000","last_became_peered":"2023-10-18T17:58:23.126114+0000","last_unstale":"2023-10-18T17:58:23.128937+0000","last_undegraded":"2023-10-18T17:58:23.128937+0000","last_fullsized":"2023-10-18T17:58:23.128937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119826+0000","last_change":"2023-10-18T17:58:22.690514+0000","last_active":"2023-10-18T17:58:23.119826+0000","last_peered":"2023-10-18T17:58:23.119826+0000","last_clean":"2023-10-18T17:58:23.119826+0000","last_became_active":"2023-10-18T17:58:22.689427+0000","last_became_peered":"2023-10-18T17:58:22.689427+0000","last_unstale":"2023-10-18T17:58:23.119826+0000","last_undegraded":"2023-10-18T17:58:23.119826+0000","last_fullsized":"2023-10-18T17:58:23.119826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135075+0000","last_change":"2023-10-18T17:58:23.128307+0000","last_active":"2023-10-18T17:58:23.135075+0000","last_peered":"2023-10-18T17:58:23.135075+0000","last_clean":"2023-10-18T17:58:23.135075+0000","last_became_active":"2023-10-18T17:58:23.127998+0000","last_became_peered":"2023-10-18T17:58:23.127998+0000","last_unstale":"2023-10-18T17:58:23.135075+0000","last_undegraded":"2023-10-18T17:58:23.135075+0000","last_fullsized":"2023-10-18T17:58:23.135075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104658+0000","last_change":"2023-10-18T17:58:22.686163+0000","last_active":"2023-10-18T17:58:23.104658+0000","last_peered":"2023-10-18T17:58:23.104658+0000","last_clean":"2023-10-18T17:58:23.104658+0000","last_became_active":"2023-10-18T17:58:22.685950+0000","last_became_peered":"2023-10-18T17:58:22.685950+0000","last_unstale":"2023-10-18T17:58:23.104658+0000","last_undegraded":"2023-10-18T17:58:23.104658+0000","last_fullsized":"2023-10-18T17:58:23.104658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125096+0000","last_change":"2023-10-18T17:58:23.123007+0000","last_active":"2023-10-18T17:58:23.125096+0000","last_peered":"2023-10-18T17:58:23.125096+0000","last_clean":"2023-10-18T17:58:23.125096+0000","last_became_active":"2023-10-18T17:58:23.122656+0000","last_became_peered":"2023-10-18T17:58:23.122656+0000","last_unstale":"2023-10-18T17:58:23.125096+0000","last_undegraded":"2023-10-18T17:58:23.125096+0000","last_fullsized":"2023-10-18T17:58:23.125096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103750+0000","last_change":"2023-10-18T17:58:22.690142+0000","last_active":"2023-10-18T17:58:23.103750+0000","last_peered":"2023-10-18T17:58:23.103750+0000","last_clean":"2023-10-18T17:58:23.103750+0000","last_became_active":"2023-10-18T17:58:22.689829+0000","last_became_peered":"2023-10-18T17:58:22.689829+0000","last_unstale":"2023-10-18T17:58:23.103750+0000","last_undegraded":"2023-10-18T17:58:23.103750+0000","last_fullsized":"2023-10-18T17:58:23.103750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.132030+0000","last_change":"2023-10-18T17:58:23.126572+0000","last_active":"2023-10-18T17:58:23.132030+0000","last_peered":"2023-10-18T17:58:23.132030+0000","last_clean":"2023-10-18T17:58:23.132030+0000","last_became_active":"2023-10-18T17:58:23.125662+0000","last_became_peered":"2023-10-18T17:58:23.125662+0000","last_unstale":"2023-10-18T17:58:23.132030+0000","last_undegraded":"2023-10-18T17:58:23.132030+0000","last_fullsized":"2023-10-18T17:58:23.132030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.103952+0000","last_change":"2023-10-18T17:58:22.674168+0000","last_active":"2023-10-18T17:58:23.103952+0000","last_peered":"2023-10-18T17:58:23.103952+0000","last_clean":"2023-10-18T17:58:23.103952+0000","last_became_active":"2023-10-18T17:58:22.668382+0000","last_became_peered":"2023-10-18T17:58:22.668382+0000","last_unstale":"2023-10-18T17:58:23.103952+0000","last_undegraded":"2023-10-18T17:58:23.103952+0000","last_fullsized":"2023-10-18T17:58:23.103952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121427+0000","last_change":"2023-10-18T17:58:23.118550+0000","last_active":"2023-10-18T17:58:23.121427+0000","last_peered":"2023-10-18T17:58:23.121427+0000","last_clean":"2023-10-18T17:58:23.121427+0000","last_became_active":"2023-10-18T17:58:23.118162+0000","last_became_peered":"2023-10-18T17:58:23.118162+0000","last_unstale":"2023-10-18T17:58:23.121427+0000","last_undegraded":"2023-10-18T17:58:23.121427+0000","last_fullsized":"2023-10-18T17:58:23.121427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.105030+0000","last_change":"2023-10-18T17:58:22.678474+0000","last_active":"2023-10-18T17:58:23.105030+0000","last_peered":"2023-10-18T17:58:23.105030+0000","last_clean":"2023-10-18T17:58:23.105030+0000","last_became_active":"2023-10-18T17:58:22.678197+0000","last_became_peered":"2023-10-18T17:58:22.678197+0000","last_unstale":"2023-10-18T17:58:23.105030+0000","last_undegraded":"2023-10-18T17:58:23.105030+0000","last_fullsized":"2023-10-18T17:58:23.105030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128857+0000","last_change":"2023-10-18T17:58:23.120777+0000","last_active":"2023-10-18T17:58:23.128857+0000","last_peered":"2023-10-18T17:58:23.128857+0000","last_clean":"2023-10-18T17:58:23.128857+0000","last_became_active":"2023-10-18T17:58:23.116893+0000","last_became_peered":"2023-10-18T17:58:23.116893+0000","last_unstale":"2023-10-18T17:58:23.128857+0000","last_undegraded":"2023-10-18T17:58:23.128857+0000","last_fullsized":"2023-10-18T17:58:23.128857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104747+0000","last_change":"2023-10-18T17:58:22.677025+0000","last_active":"2023-10-18T17:58:23.104747+0000","last_peered":"2023-10-18T17:58:23.104747+0000","last_clean":"2023-10-18T17:58:23.104747+0000","last_became_active":"2023-10-18T17:58:22.676536+0000","last_became_peered":"2023-10-18T17:58:22.676536+0000","last_unstale":"2023-10-18T17:58:23.104747+0000","last_undegraded":"2023-10-18T17:58:23.104747+0000","last_fullsized":"2023-10-18T17:58:23.104747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129785+0000","last_change":"2023-10-18T17:58:23.121230+0000","last_active":"2023-10-18T17:58:23.129785+0000","last_peered":"2023-10-18T17:58:23.129785+0000","last_clean":"2023-10-18T17:58:23.129785+0000","last_became_active":"2023-10-18T17:58:23.120977+0000","last_became_peered":"2023-10-18T17:58:23.120977+0000","last_unstale":"2023-10-18T17:58:23.129785+0000","last_undegraded":"2023-10-18T17:58:23.129785+0000","last_fullsized":"2023-10-18T17:58:23.129785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.107108+0000","last_change":"2023-10-18T17:58:22.691513+0000","last_active":"2023-10-18T17:58:23.107108+0000","last_peered":"2023-10-18T17:58:23.107108+0000","last_clean":"2023-10-18T17:58:23.107108+0000","last_became_active":"2023-10-18T17:58:22.691220+0000","last_became_peered":"2023-10-18T17:58:22.691220+0000","last_unstale":"2023-10-18T17:58:23.107108+0000","last_undegraded":"2023-10-18T17:58:23.107108+0000","last_fullsized":"2023-10-18T17:58:23.107108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.131817+0000","last_change":"2023-10-18T17:58:23.128454+0000","last_active":"2023-10-18T17:58:23.131817+0000","last_peered":"2023-10-18T17:58:23.131817+0000","last_clean":"2023-10-18T17:58:23.131817+0000","last_became_active":"2023-10-18T17:58:23.128113+0000","last_became_peered":"2023-10-18T17:58:23.128113+0000","last_unstale":"2023-10-18T17:58:23.131817+0000","last_undegraded":"2023-10-18T17:58:23.131817+0000","last_fullsized":"2023-10-18T17:58:23.131817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116763+0000","last_change":"2023-10-18T17:58:22.695883+0000","last_active":"2023-10-18T17:58:23.116763+0000","last_peered":"2023-10-18T17:58:23.116763+0000","last_clean":"2023-10-18T17:58:23.116763+0000","last_became_active":"2023-10-18T17:58:22.695183+0000","last_became_peered":"2023-10-18T17:58:22.695183+0000","last_unstale":"2023-10-18T17:58:23.116763+0000","last_undegraded":"2023-10-18T17:58:23.116763+0000","last_fullsized":"2023-10-18T17:58:23.116763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.131910+0000","last_change":"2023-10-18T17:58:23.123828+0000","last_active":"2023-10-18T17:58:23.131910+0000","last_peered":"2023-10-18T17:58:23.131910+0000","last_clean":"2023-10-18T17:58:23.131910+0000","last_became_active":"2023-10-18T17:58:23.123552+0000","last_became_peered":"2023-10-18T17:58:23.123552+0000","last_unstale":"2023-10-18T17:58:23.131910+0000","last_undegraded":"2023-10-18T17:58:23.131910+0000","last_fullsized":"2023-10-18T17:58:23.131910+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111182+0000","last_change":"2023-10-18T17:58:22.685450+0000","last_active":"2023-10-18T17:58:23.111182+0000","last_peered":"2023-10-18T17:58:23.111182+0000","last_clean":"2023-10-18T17:58:23.111182+0000","last_became_active":"2023-10-18T17:58:22.685168+0000","last_became_peered":"2023-10-18T17:58:22.685168+0000","last_unstale":"2023-10-18T17:58:23.111182+0000","last_undegraded":"2023-10-18T17:58:23.111182+0000","last_fullsized":"2023-10-18T17:58:23.111182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120499+0000","last_change":"2023-10-18T17:58:23.114901+0000","last_active":"2023-10-18T17:58:23.120499+0000","last_peered":"2023-10-18T17:58:23.120499+0000","last_clean":"2023-10-18T17:58:23.120499+0000","last_became_active":"2023-10-18T17:58:23.114590+0000","last_became_peered":"2023-10-18T17:58:23.114590+0000","last_unstale":"2023-10-18T17:58:23.120499+0000","last_undegraded":"2023-10-18T17:58:23.120499+0000","last_fullsized":"2023-10-18T17:58:23.120499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112133+0000","last_change":"2023-10-18T17:58:22.692159+0000","last_active":"2023-10-18T17:58:23.112133+0000","last_peered":"2023-10-18T17:58:23.112133+0000","last_clean":"2023-10-18T17:58:23.112133+0000","last_became_active":"2023-10-18T17:58:22.691846+0000","last_became_peered":"2023-10-18T17:58:22.691846+0000","last_unstale":"2023-10-18T17:58:23.112133+0000","last_undegraded":"2023-10-18T17:58:23.112133+0000","last_fullsized":"2023-10-18T17:58:23.112133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136653+0000","last_change":"2023-10-18T17:58:23.134892+0000","last_active":"2023-10-18T17:58:23.136653+0000","last_peered":"2023-10-18T17:58:23.136653+0000","last_clean":"2023-10-18T17:58:23.136653+0000","last_became_active":"2023-10-18T17:58:23.134213+0000","last_became_peered":"2023-10-18T17:58:23.134213+0000","last_unstale":"2023-10-18T17:58:23.136653+0000","last_undegraded":"2023-10-18T17:58:23.136653+0000","last_fullsized":"2023-10-18T17:58:23.136653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120693+0000","last_change":"2023-10-18T17:58:17.791475+0000","last_active":"2023-10-18T17:58:23.120693+0000","last_peered":"2023-10-18T17:58:23.120693+0000","last_clean":"2023-10-18T17:58:23.120693+0000","last_became_active":"2023-10-18T17:58:17.790774+0000","last_became_peered":"2023-10-18T17:58:17.790774+0000","last_unstale":"2023-10-18T17:58:23.120693+0000","last_undegraded":"2023-10-18T17:58:23.120693+0000","last_fullsized":"2023-10-18T17:58:23.120693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104540+0000","last_change":"2023-10-18T17:58:22.676456+0000","last_active":"2023-10-18T17:58:23.104540+0000","last_peered":"2023-10-18T17:58:23.104540+0000","last_clean":"2023-10-18T17:58:23.104540+0000","last_became_active":"2023-10-18T17:58:22.676189+0000","last_became_peered":"2023-10-18T17:58:22.676189+0000","last_unstale":"2023-10-18T17:58:23.104540+0000","last_undegraded":"2023-10-18T17:58:23.104540+0000","last_fullsized":"2023-10-18T17:58:23.104540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136521+0000","last_change":"2023-10-18T17:58:23.133571+0000","last_active":"2023-10-18T17:58:23.136521+0000","last_peered":"2023-10-18T17:58:23.136521+0000","last_clean":"2023-10-18T17:58:23.136521+0000","last_became_active":"2023-10-18T17:58:23.133302+0000","last_became_peered":"2023-10-18T17:58:23.133302+0000","last_unstale":"2023-10-18T17:58:23.136521+0000","last_undegraded":"2023-10-18T17:58:23.136521+0000","last_fullsized":"2023-10-18T17:58:23.136521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104626+0000","last_change":"2023-10-18T17:58:17.792368+0000","last_active":"2023-10-18T17:58:23.104626+0000","last_peered":"2023-10-18T17:58:23.104626+0000","last_clean":"2023-10-18T17:58:23.104626+0000","last_became_active":"2023-10-18T17:58:17.791751+0000","last_became_peered":"2023-10-18T17:58:17.791751+0000","last_unstale":"2023-10-18T17:58:23.104626+0000","last_undegraded":"2023-10-18T17:58:23.104626+0000","last_fullsized":"2023-10-18T17:58:23.104626+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110468+0000","last_change":"2023-10-18T17:58:22.675348+0000","last_active":"2023-10-18T17:58:23.110468+0000","last_peered":"2023-10-18T17:58:23.110468+0000","last_clean":"2023-10-18T17:58:23.110468+0000","last_became_active":"2023-10-18T17:58:22.674469+0000","last_became_peered":"2023-10-18T17:58:22.674469+0000","last_unstale":"2023-10-18T17:58:23.110468+0000","last_undegraded":"2023-10-18T17:58:23.110468+0000","last_fullsized":"2023-10-18T17:58:23.110468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128966+0000","last_change":"2023-10-18T17:58:23.121217+0000","last_active":"2023-10-18T17:58:23.128966+0000","last_peered":"2023-10-18T17:58:23.128966+0000","last_clean":"2023-10-18T17:58:23.128966+0000","last_became_active":"2023-10-18T17:58:23.120691+0000","last_became_peered":"2023-10-18T17:58:23.120691+0000","last_unstale":"2023-10-18T17:58:23.128966+0000","last_undegraded":"2023-10-18T17:58:23.128966+0000","last_fullsized":"2023-10-18T17:58:23.128966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120107+0000","last_change":"2023-10-18T17:58:17.605810+0000","last_active":"2023-10-18T17:58:23.120107+0000","last_peered":"2023-10-18T17:58:23.120107+0000","last_clean":"2023-10-18T17:58:23.120107+0000","last_became_active":"2023-10-18T17:58:17.605289+0000","last_became_peered":"2023-10-18T17:58:17.605289+0000","last_unstale":"2023-10-18T17:58:23.120107+0000","last_undegraded":"2023-10-18T17:58:23.120107+0000","last_fullsized":"2023-10-18T17:58:23.120107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118076+0000","last_change":"2023-10-18T17:58:22.675274+0000","last_active":"2023-10-18T17:58:23.118076+0000","last_peered":"2023-10-18T17:58:23.118076+0000","last_clean":"2023-10-18T17:58:23.118076+0000","last_became_active":"2023-10-18T17:58:22.674965+0000","last_became_peered":"2023-10-18T17:58:22.674965+0000","last_unstale":"2023-10-18T17:58:23.118076+0000","last_undegraded":"2023-10-18T17:58:23.118076+0000","last_fullsized":"2023-10-18T17:58:23.118076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129425+0000","last_change":"2023-10-18T17:58:23.125547+0000","last_active":"2023-10-18T17:58:23.129425+0000","last_peered":"2023-10-18T17:58:23.129425+0000","last_clean":"2023-10-18T17:58:23.129425+0000","last_became_active":"2023-10-18T17:58:23.125259+0000","last_became_peered":"2023-10-18T17:58:23.125259+0000","last_unstale":"2023-10-18T17:58:23.129425+0000","last_undegraded":"2023-10-18T17:58:23.129425+0000","last_fullsized":"2023-10-18T17:58:23.129425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.106468+0000","last_change":"2023-10-18T17:58:18.326048+0000","last_active":"2023-10-18T17:58:23.106468+0000","last_peered":"2023-10-18T17:58:23.106468+0000","last_clean":"2023-10-18T17:58:23.106468+0000","last_became_active":"2023-10-18T17:58:18.325141+0000","last_became_peered":"2023-10-18T17:58:18.325141+0000","last_unstale":"2023-10-18T17:58:23.106468+0000","last_undegraded":"2023-10-18T17:58:23.106468+0000","last_fullsized":"2023-10-18T17:58:23.106468+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114331+0000","last_change":"2023-10-18T17:58:22.684816+0000","last_active":"2023-10-18T17:58:23.114331+0000","last_peered":"2023-10-18T17:58:23.114331+0000","last_clean":"2023-10-18T17:58:23.114331+0000","last_became_active":"2023-10-18T17:58:22.682458+0000","last_became_peered":"2023-10-18T17:58:22.682458+0000","last_unstale":"2023-10-18T17:58:23.114331+0000","last_undegraded":"2023-10-18T17:58:23.114331+0000","last_fullsized":"2023-10-18T17:58:23.114331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134601+0000","last_change":"2023-10-18T17:58:23.131050+0000","last_active":"2023-10-18T17:58:23.134601+0000","last_peered":"2023-10-18T17:58:23.134601+0000","last_clean":"2023-10-18T17:58:23.134601+0000","last_became_active":"2023-10-18T17:58:23.130768+0000","last_became_peered":"2023-10-18T17:58:23.130768+0000","last_unstale":"2023-10-18T17:58:23.134601+0000","last_undegraded":"2023-10-18T17:58:23.134601+0000","last_fullsized":"2023-10-18T17:58:23.134601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110585+0000","last_change":"2023-10-18T17:58:22.698814+0000","last_active":"2023-10-18T17:58:23.110585+0000","last_peered":"2023-10-18T17:58:23.110585+0000","last_clean":"2023-10-18T17:58:23.110585+0000","last_became_active":"2023-10-18T17:58:22.698525+0000","last_became_peered":"2023-10-18T17:58:22.698525+0000","last_unstale":"2023-10-18T17:58:23.110585+0000","last_undegraded":"2023-10-18T17:58:23.110585+0000","last_fullsized":"2023-10-18T17:58:23.110585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117324+0000","last_change":"2023-10-18T17:58:18.322815+0000","last_active":"2023-10-18T17:58:23.117324+0000","last_peered":"2023-10-18T17:58:23.117324+0000","last_clean":"2023-10-18T17:58:23.117324+0000","last_became_active":"2023-10-18T17:58:18.322057+0000","last_became_peered":"2023-10-18T17:58:18.322057+0000","last_unstale":"2023-10-18T17:58:23.117324+0000","last_undegraded":"2023-10-18T17:58:23.117324+0000","last_fullsized":"2023-10-18T17:58:23.117324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.130103+0000","last_change":"2023-10-18T17:58:23.125081+0000","last_active":"2023-10-18T17:58:23.130103+0000","last_peered":"2023-10-18T17:58:23.130103+0000","last_clean":"2023-10-18T17:58:23.130103+0000","last_became_active":"2023-10-18T17:58:23.124833+0000","last_became_peered":"2023-10-18T17:58:23.124833+0000","last_unstale":"2023-10-18T17:58:23.130103+0000","last_undegraded":"2023-10-18T17:58:23.130103+0000","last_fullsized":"2023-10-18T17:58:23.130103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104554+0000","last_change":"2023-10-18T17:58:22.680737+0000","last_active":"2023-10-18T17:58:23.104554+0000","last_peered":"2023-10-18T17:58:23.104554+0000","last_clean":"2023-10-18T17:58:23.104554+0000","last_became_active":"2023-10-18T17:58:22.676981+0000","last_became_peered":"2023-10-18T17:58:22.676981+0000","last_unstale":"2023-10-18T17:58:23.104554+0000","last_undegraded":"2023-10-18T17:58:23.104554+0000","last_fullsized":"2023-10-18T17:58:23.104554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.108047+0000","last_change":"2023-10-18T17:58:17.608107+0000","last_active":"2023-10-18T17:58:23.108047+0000","last_peered":"2023-10-18T17:58:23.108047+0000","last_clean":"2023-10-18T17:58:23.108047+0000","last_became_active":"2023-10-18T17:58:17.607603+0000","last_became_peered":"2023-10-18T17:58:17.607603+0000","last_unstale":"2023-10-18T17:58:23.108047+0000","last_undegraded":"2023-10-18T17:58:23.108047+0000","last_fullsized":"2023-10-18T17:58:23.108047+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129444+0000","last_change":"2023-10-18T17:58:23.125950+0000","last_active":"2023-10-18T17:58:23.129444+0000","last_peered":"2023-10-18T17:58:23.129444+0000","last_clean":"2023-10-18T17:58:23.129444+0000","last_became_active":"2023-10-18T17:58:23.125172+0000","last_became_peered":"2023-10-18T17:58:23.125172+0000","last_unstale":"2023-10-18T17:58:23.129444+0000","last_undegraded":"2023-10-18T17:58:23.129444+0000","last_fullsized":"2023-10-18T17:58:23.129444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117966+0000","last_change":"2023-10-18T17:58:22.683261+0000","last_active":"2023-10-18T17:58:23.117966+0000","last_peered":"2023-10-18T17:58:23.117966+0000","last_clean":"2023-10-18T17:58:23.117966+0000","last_became_active":"2023-10-18T17:58:22.682927+0000","last_became_peered":"2023-10-18T17:58:22.682927+0000","last_unstale":"2023-10-18T17:58:23.117966+0000","last_undegraded":"2023-10-18T17:58:23.117966+0000","last_fullsized":"2023-10-18T17:58:23.117966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.114484+0000","last_change":"2023-10-18T17:58:17.607313+0000","last_active":"2023-10-18T17:58:23.114484+0000","last_peered":"2023-10-18T17:58:23.114484+0000","last_clean":"2023-10-18T17:58:23.114484+0000","last_became_active":"2023-10-18T17:58:17.606933+0000","last_became_peered":"2023-10-18T17:58:17.606933+0000","last_unstale":"2023-10-18T17:58:23.114484+0000","last_undegraded":"2023-10-18T17:58:23.114484+0000","last_fullsized":"2023-10-18T17:58:23.114484+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118875+0000","last_change":"2023-10-18T17:58:23.116455+0000","last_active":"2023-10-18T17:58:23.118875+0000","last_peered":"2023-10-18T17:58:23.118875+0000","last_clean":"2023-10-18T17:58:23.118875+0000","last_became_active":"2023-10-18T17:58:23.116050+0000","last_became_peered":"2023-10-18T17:58:23.116050+0000","last_unstale":"2023-10-18T17:58:23.118875+0000","last_undegraded":"2023-10-18T17:58:23.118875+0000","last_fullsized":"2023-10-18T17:58:23.118875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110614+0000","last_change":"2023-10-18T17:58:22.689750+0000","last_active":"2023-10-18T17:58:23.110614+0000","last_peered":"2023-10-18T17:58:23.110614+0000","last_clean":"2023-10-18T17:58:23.110614+0000","last_became_active":"2023-10-18T17:58:22.689474+0000","last_became_peered":"2023-10-18T17:58:22.689474+0000","last_unstale":"2023-10-18T17:58:23.110614+0000","last_undegraded":"2023-10-18T17:58:23.110614+0000","last_fullsized":"2023-10-18T17:58:23.110614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112122+0000","last_change":"2023-10-18T17:58:17.646977+0000","last_active":"2023-10-18T17:58:23.112122+0000","last_peered":"2023-10-18T17:58:23.112122+0000","last_clean":"2023-10-18T17:58:23.112122+0000","last_became_active":"2023-10-18T17:58:17.646568+0000","last_became_peered":"2023-10-18T17:58:17.646568+0000","last_unstale":"2023-10-18T17:58:23.112122+0000","last_undegraded":"2023-10-18T17:58:23.112122+0000","last_fullsized":"2023-10-18T17:58:23.112122+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:16.579089+0000","last_clean_scrub_stamp":"2023-10-18T17:58:16.579089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.101935+0000","last_change":"2023-10-18T17:58:15.581426+0000","last_active":"2023-10-18T17:58:23.101935+0000","last_peered":"2023-10-18T17:58:23.101935+0000","last_clean":"2023-10-18T17:58:23.101935+0000","last_became_active":"2023-10-18T17:58:15.580900+0000","last_became_peered":"2023-10-18T17:58:15.580900+0000","last_unstale":"2023-10-18T17:58:23.101935+0000","last_undegraded":"2023-10-18T17:58:23.101935+0000","last_fullsized":"2023-10-18T17:58:23.101935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:14.550597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:14.550597+0000","last_clean_scrub_stamp":"2023-10-18T17:58:14.550597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135976+0000","last_change":"2023-10-18T17:58:23.130365+0000","last_active":"2023-10-18T17:58:23.135976+0000","last_peered":"2023-10-18T17:58:23.135976+0000","last_clean":"2023-10-18T17:58:23.135976+0000","last_became_active":"2023-10-18T17:58:23.128150+0000","last_became_peered":"2023-10-18T17:58:23.128150+0000","last_unstale":"2023-10-18T17:58:23.135976+0000","last_undegraded":"2023-10-18T17:58:23.135976+0000","last_fullsized":"2023-10-18T17:58:23.135976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110186+0000","last_change":"2023-10-18T17:58:22.675737+0000","last_active":"2023-10-18T17:58:23.110186+0000","last_peered":"2023-10-18T17:58:23.110186+0000","last_clean":"2023-10-18T17:58:23.110186+0000","last_became_active":"2023-10-18T17:58:22.675306+0000","last_became_peered":"2023-10-18T17:58:22.675306+0000","last_unstale":"2023-10-18T17:58:23.110186+0000","last_undegraded":"2023-10-18T17:58:23.110186+0000","last_fullsized":"2023-10-18T17:58:23.110186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119262+0000","last_change":"2023-10-18T17:58:23.115745+0000","last_active":"2023-10-18T17:58:23.119262+0000","last_peered":"2023-10-18T17:58:23.119262+0000","last_clean":"2023-10-18T17:58:23.119262+0000","last_became_active":"2023-10-18T17:58:23.115454+0000","last_became_peered":"2023-10-18T17:58:23.115454+0000","last_unstale":"2023-10-18T17:58:23.119262+0000","last_undegraded":"2023-10-18T17:58:23.119262+0000","last_fullsized":"2023-10-18T17:58:23.119262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.113848+0000","last_change":"2023-10-18T17:58:22.690810+0000","last_active":"2023-10-18T17:58:23.113848+0000","last_peered":"2023-10-18T17:58:23.113848+0000","last_clean":"2023-10-18T17:58:23.113848+0000","last_became_active":"2023-10-18T17:58:22.690506+0000","last_became_peered":"2023-10-18T17:58:22.690506+0000","last_unstale":"2023-10-18T17:58:23.113848+0000","last_undegraded":"2023-10-18T17:58:23.113848+0000","last_fullsized":"2023-10-18T17:58:23.113848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.133051+0000","last_change":"2023-10-18T17:58:23.126643+0000","last_active":"2023-10-18T17:58:23.133051+0000","last_peered":"2023-10-18T17:58:23.133051+0000","last_clean":"2023-10-18T17:58:23.133051+0000","last_became_active":"2023-10-18T17:58:23.126371+0000","last_became_peered":"2023-10-18T17:58:23.126371+0000","last_unstale":"2023-10-18T17:58:23.133051+0000","last_undegraded":"2023-10-18T17:58:23.133051+0000","last_fullsized":"2023-10-18T17:58:23.133051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117250+0000","last_change":"2023-10-18T17:58:22.690893+0000","last_active":"2023-10-18T17:58:23.117250+0000","last_peered":"2023-10-18T17:58:23.117250+0000","last_clean":"2023-10-18T17:58:23.117250+0000","last_became_active":"2023-10-18T17:58:22.689078+0000","last_became_peered":"2023-10-18T17:58:22.689078+0000","last_unstale":"2023-10-18T17:58:23.117250+0000","last_undegraded":"2023-10-18T17:58:23.117250+0000","last_fullsized":"2023-10-18T17:58:23.117250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135993+0000","last_change":"2023-10-18T17:58:23.130578+0000","last_active":"2023-10-18T17:58:23.135993+0000","last_peered":"2023-10-18T17:58:23.135993+0000","last_clean":"2023-10-18T17:58:23.135993+0000","last_became_active":"2023-10-18T17:58:23.130103+0000","last_became_peered":"2023-10-18T17:58:23.130103+0000","last_unstale":"2023-10-18T17:58:23.135993+0000","last_undegraded":"2023-10-18T17:58:23.135993+0000","last_fullsized":"2023-10-18T17:58:23.135993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.108236+0000","last_change":"2023-10-18T17:58:22.682353+0000","last_active":"2023-10-18T17:58:23.108236+0000","last_peered":"2023-10-18T17:58:23.108236+0000","last_clean":"2023-10-18T17:58:23.108236+0000","last_became_active":"2023-10-18T17:58:22.682040+0000","last_became_peered":"2023-10-18T17:58:22.682040+0000","last_unstale":"2023-10-18T17:58:23.108236+0000","last_undegraded":"2023-10-18T17:58:23.108236+0000","last_fullsized":"2023-10-18T17:58:23.108236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127708+0000","last_change":"2023-10-18T17:58:23.120218+0000","last_active":"2023-10-18T17:58:23.127708+0000","last_peered":"2023-10-18T17:58:23.127708+0000","last_clean":"2023-10-18T17:58:23.127708+0000","last_became_active":"2023-10-18T17:58:23.119968+0000","last_became_peered":"2023-10-18T17:58:23.119968+0000","last_unstale":"2023-10-18T17:58:23.127708+0000","last_undegraded":"2023-10-18T17:58:23.127708+0000","last_fullsized":"2023-10-18T17:58:23.127708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110949+0000","last_change":"2023-10-18T17:58:22.679610+0000","last_active":"2023-10-18T17:58:23.110949+0000","last_peered":"2023-10-18T17:58:23.110949+0000","last_clean":"2023-10-18T17:58:23.110949+0000","last_became_active":"2023-10-18T17:58:22.678880+0000","last_became_peered":"2023-10-18T17:58:22.678880+0000","last_unstale":"2023-10-18T17:58:23.110949+0000","last_undegraded":"2023-10-18T17:58:23.110949+0000","last_fullsized":"2023-10-18T17:58:23.110949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.120763+0000","last_change":"2023-10-18T17:58:23.118035+0000","last_active":"2023-10-18T17:58:23.120763+0000","last_peered":"2023-10-18T17:58:23.120763+0000","last_clean":"2023-10-18T17:58:23.120763+0000","last_became_active":"2023-10-18T17:58:23.117448+0000","last_became_peered":"2023-10-18T17:58:23.117448+0000","last_unstale":"2023-10-18T17:58:23.120763+0000","last_undegraded":"2023-10-18T17:58:23.120763+0000","last_fullsized":"2023-10-18T17:58:23.120763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.119936+0000","last_change":"2023-10-18T17:58:22.688810+0000","last_active":"2023-10-18T17:58:23.119936+0000","last_peered":"2023-10-18T17:58:23.119936+0000","last_clean":"2023-10-18T17:58:23.119936+0000","last_became_active":"2023-10-18T17:58:22.688224+0000","last_became_peered":"2023-10-18T17:58:22.688224+0000","last_unstale":"2023-10-18T17:58:23.119936+0000","last_undegraded":"2023-10-18T17:58:23.119936+0000","last_fullsized":"2023-10-18T17:58:23.119936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.136556+0000","last_change":"2023-10-18T17:58:23.133896+0000","last_active":"2023-10-18T17:58:23.136556+0000","last_peered":"2023-10-18T17:58:23.136556+0000","last_clean":"2023-10-18T17:58:23.136556+0000","last_became_active":"2023-10-18T17:58:23.133588+0000","last_became_peered":"2023-10-18T17:58:23.133588+0000","last_unstale":"2023-10-18T17:58:23.136556+0000","last_undegraded":"2023-10-18T17:58:23.136556+0000","last_fullsized":"2023-10-18T17:58:23.136556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116851+0000","last_change":"2023-10-18T17:58:22.696491+0000","last_active":"2023-10-18T17:58:23.116851+0000","last_peered":"2023-10-18T17:58:23.116851+0000","last_clean":"2023-10-18T17:58:23.116851+0000","last_became_active":"2023-10-18T17:58:22.696171+0000","last_became_peered":"2023-10-18T17:58:22.696171+0000","last_unstale":"2023-10-18T17:58:23.116851+0000","last_undegraded":"2023-10-18T17:58:23.116851+0000","last_fullsized":"2023-10-18T17:58:23.116851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.125405+0000","last_change":"2023-10-18T17:58:23.116692+0000","last_active":"2023-10-18T17:58:23.125405+0000","last_peered":"2023-10-18T17:58:23.125405+0000","last_clean":"2023-10-18T17:58:23.125405+0000","last_became_active":"2023-10-18T17:58:23.116431+0000","last_became_peered":"2023-10-18T17:58:23.116431+0000","last_unstale":"2023-10-18T17:58:23.125405+0000","last_undegraded":"2023-10-18T17:58:23.125405+0000","last_fullsized":"2023-10-18T17:58:23.125405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111957+0000","last_change":"2023-10-18T17:58:22.694825+0000","last_active":"2023-10-18T17:58:23.111957+0000","last_peered":"2023-10-18T17:58:23.111957+0000","last_clean":"2023-10-18T17:58:23.111957+0000","last_became_active":"2023-10-18T17:58:22.694563+0000","last_became_peered":"2023-10-18T17:58:22.694563+0000","last_unstale":"2023-10-18T17:58:23.111957+0000","last_undegraded":"2023-10-18T17:58:23.111957+0000","last_fullsized":"2023-10-18T17:58:23.111957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126937+0000","last_change":"2023-10-18T17:58:23.120769+0000","last_active":"2023-10-18T17:58:23.126937+0000","last_peered":"2023-10-18T17:58:23.126937+0000","last_clean":"2023-10-18T17:58:23.126937+0000","last_became_active":"2023-10-18T17:58:23.120464+0000","last_became_peered":"2023-10-18T17:58:23.120464+0000","last_unstale":"2023-10-18T17:58:23.126937+0000","last_undegraded":"2023-10-18T17:58:23.126937+0000","last_fullsized":"2023-10-18T17:58:23.126937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.101806+0000","last_change":"2023-10-18T17:58:22.668326+0000","last_active":"2023-10-18T17:58:23.101806+0000","last_peered":"2023-10-18T17:58:23.101806+0000","last_clean":"2023-10-18T17:58:23.101806+0000","last_became_active":"2023-10-18T17:58:22.668059+0000","last_became_peered":"2023-10-18T17:58:22.668059+0000","last_unstale":"2023-10-18T17:58:23.101806+0000","last_undegraded":"2023-10-18T17:58:23.101806+0000","last_fullsized":"2023-10-18T17:58:23.101806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.124121+0000","last_change":"2023-10-18T17:58:23.120596+0000","last_active":"2023-10-18T17:58:23.124121+0000","last_peered":"2023-10-18T17:58:23.124121+0000","last_clean":"2023-10-18T17:58:23.124121+0000","last_became_active":"2023-10-18T17:58:23.120309+0000","last_became_peered":"2023-10-18T17:58:23.120309+0000","last_unstale":"2023-10-18T17:58:23.124121+0000","last_undegraded":"2023-10-18T17:58:23.124121+0000","last_fullsized":"2023-10-18T17:58:23.124121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.113755+0000","last_change":"2023-10-18T17:58:22.700021+0000","last_active":"2023-10-18T17:58:23.113755+0000","last_peered":"2023-10-18T17:58:23.113755+0000","last_clean":"2023-10-18T17:58:23.113755+0000","last_became_active":"2023-10-18T17:58:22.699570+0000","last_became_peered":"2023-10-18T17:58:22.699570+0000","last_unstale":"2023-10-18T17:58:23.113755+0000","last_undegraded":"2023-10-18T17:58:23.113755+0000","last_fullsized":"2023-10-18T17:58:23.113755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.121251+0000","last_change":"2023-10-18T17:58:23.118771+0000","last_active":"2023-10-18T17:58:23.121251+0000","last_peered":"2023-10-18T17:58:23.121251+0000","last_clean":"2023-10-18T17:58:23.121251+0000","last_became_active":"2023-10-18T17:58:23.118493+0000","last_became_peered":"2023-10-18T17:58:23.118493+0000","last_unstale":"2023-10-18T17:58:23.121251+0000","last_undegraded":"2023-10-18T17:58:23.121251+0000","last_fullsized":"2023-10-18T17:58:23.121251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117844+0000","last_change":"2023-10-18T17:58:22.694576+0000","last_active":"2023-10-18T17:58:23.117844+0000","last_peered":"2023-10-18T17:58:23.117844+0000","last_clean":"2023-10-18T17:58:23.117844+0000","last_became_active":"2023-10-18T17:58:22.694240+0000","last_became_peered":"2023-10-18T17:58:22.694240+0000","last_unstale":"2023-10-18T17:58:23.117844+0000","last_undegraded":"2023-10-18T17:58:23.117844+0000","last_fullsized":"2023-10-18T17:58:23.117844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134198+0000","last_change":"2023-10-18T17:58:23.126726+0000","last_active":"2023-10-18T17:58:23.134198+0000","last_peered":"2023-10-18T17:58:23.134198+0000","last_clean":"2023-10-18T17:58:23.134198+0000","last_became_active":"2023-10-18T17:58:23.126297+0000","last_became_peered":"2023-10-18T17:58:23.126297+0000","last_unstale":"2023-10-18T17:58:23.134198+0000","last_undegraded":"2023-10-18T17:58:23.134198+0000","last_fullsized":"2023-10-18T17:58:23.134198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.112044+0000","last_change":"2023-10-18T17:58:22.690048+0000","last_active":"2023-10-18T17:58:23.112044+0000","last_peered":"2023-10-18T17:58:23.112044+0000","last_clean":"2023-10-18T17:58:23.112044+0000","last_became_active":"2023-10-18T17:58:22.689785+0000","last_became_peered":"2023-10-18T17:58:22.689785+0000","last_unstale":"2023-10-18T17:58:23.112044+0000","last_undegraded":"2023-10-18T17:58:23.112044+0000","last_fullsized":"2023-10-18T17:58:23.112044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.134590+0000","last_change":"2023-10-18T17:58:23.129919+0000","last_active":"2023-10-18T17:58:23.134590+0000","last_peered":"2023-10-18T17:58:23.134590+0000","last_clean":"2023-10-18T17:58:23.134590+0000","last_became_active":"2023-10-18T17:58:23.129560+0000","last_became_peered":"2023-10-18T17:58:23.129560+0000","last_unstale":"2023-10-18T17:58:23.134590+0000","last_undegraded":"2023-10-18T17:58:23.134590+0000","last_fullsized":"2023-10-18T17:58:23.134590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118421+0000","last_change":"2023-10-18T17:58:22.678801+0000","last_active":"2023-10-18T17:58:23.118421+0000","last_peered":"2023-10-18T17:58:23.118421+0000","last_clean":"2023-10-18T17:58:23.118421+0000","last_became_active":"2023-10-18T17:58:22.678493+0000","last_became_peered":"2023-10-18T17:58:22.678493+0000","last_unstale":"2023-10-18T17:58:23.118421+0000","last_undegraded":"2023-10-18T17:58:23.118421+0000","last_fullsized":"2023-10-18T17:58:23.118421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129374+0000","last_change":"2023-10-18T17:58:23.124984+0000","last_active":"2023-10-18T17:58:23.129374+0000","last_peered":"2023-10-18T17:58:23.129374+0000","last_clean":"2023-10-18T17:58:23.129374+0000","last_became_active":"2023-10-18T17:58:23.124646+0000","last_became_peered":"2023-10-18T17:58:23.124646+0000","last_unstale":"2023-10-18T17:58:23.129374+0000","last_undegraded":"2023-10-18T17:58:23.129374+0000","last_fullsized":"2023-10-18T17:58:23.129374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.118029+0000","last_change":"2023-10-18T17:58:22.677885+0000","last_active":"2023-10-18T17:58:23.118029+0000","last_peered":"2023-10-18T17:58:23.118029+0000","last_clean":"2023-10-18T17:58:23.118029+0000","last_became_active":"2023-10-18T17:58:22.677592+0000","last_became_peered":"2023-10-18T17:58:22.677592+0000","last_unstale":"2023-10-18T17:58:23.118029+0000","last_undegraded":"2023-10-18T17:58:23.118029+0000","last_fullsized":"2023-10-18T17:58:23.118029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.117238+0000","last_change":"2023-10-18T17:58:23.115290+0000","last_active":"2023-10-18T17:58:23.117238+0000","last_peered":"2023-10-18T17:58:23.117238+0000","last_clean":"2023-10-18T17:58:23.117238+0000","last_became_active":"2023-10-18T17:58:23.115014+0000","last_became_peered":"2023-10-18T17:58:23.115014+0000","last_unstale":"2023-10-18T17:58:23.117238+0000","last_undegraded":"2023-10-18T17:58:23.117238+0000","last_fullsized":"2023-10-18T17:58:23.117238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110719+0000","last_change":"2023-10-18T17:58:22.695980+0000","last_active":"2023-10-18T17:58:23.110719+0000","last_peered":"2023-10-18T17:58:23.110719+0000","last_clean":"2023-10-18T17:58:23.110719+0000","last_became_active":"2023-10-18T17:58:22.695592+0000","last_became_peered":"2023-10-18T17:58:22.695592+0000","last_unstale":"2023-10-18T17:58:23.110719+0000","last_undegraded":"2023-10-18T17:58:23.110719+0000","last_fullsized":"2023-10-18T17:58:23.110719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127363+0000","last_change":"2023-10-18T17:58:23.124321+0000","last_active":"2023-10-18T17:58:23.127363+0000","last_peered":"2023-10-18T17:58:23.127363+0000","last_clean":"2023-10-18T17:58:23.127363+0000","last_became_active":"2023-10-18T17:58:23.124019+0000","last_became_peered":"2023-10-18T17:58:23.124019+0000","last_unstale":"2023-10-18T17:58:23.127363+0000","last_undegraded":"2023-10-18T17:58:23.127363+0000","last_fullsized":"2023-10-18T17:58:23.127363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111016+0000","last_change":"2023-10-18T17:58:22.692184+0000","last_active":"2023-10-18T17:58:23.111016+0000","last_peered":"2023-10-18T17:58:23.111016+0000","last_clean":"2023-10-18T17:58:23.111016+0000","last_became_active":"2023-10-18T17:58:22.691802+0000","last_became_peered":"2023-10-18T17:58:22.691802+0000","last_unstale":"2023-10-18T17:58:23.111016+0000","last_undegraded":"2023-10-18T17:58:23.111016+0000","last_fullsized":"2023-10-18T17:58:23.111016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126067+0000","last_change":"2023-10-18T17:58:23.116039+0000","last_active":"2023-10-18T17:58:23.126067+0000","last_peered":"2023-10-18T17:58:23.126067+0000","last_clean":"2023-10-18T17:58:23.126067+0000","last_became_active":"2023-10-18T17:58:23.115715+0000","last_became_peered":"2023-10-18T17:58:23.115715+0000","last_unstale":"2023-10-18T17:58:23.126067+0000","last_undegraded":"2023-10-18T17:58:23.126067+0000","last_fullsized":"2023-10-18T17:58:23.126067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111682+0000","last_change":"2023-10-18T17:58:22.687555+0000","last_active":"2023-10-18T17:58:23.111682+0000","last_peered":"2023-10-18T17:58:23.111682+0000","last_clean":"2023-10-18T17:58:23.111682+0000","last_became_active":"2023-10-18T17:58:22.687269+0000","last_became_peered":"2023-10-18T17:58:22.687269+0000","last_unstale":"2023-10-18T17:58:23.111682+0000","last_undegraded":"2023-10-18T17:58:23.111682+0000","last_fullsized":"2023-10-18T17:58:23.111682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127657+0000","last_change":"2023-10-18T17:58:23.120769+0000","last_active":"2023-10-18T17:58:23.127657+0000","last_peered":"2023-10-18T17:58:23.127657+0000","last_clean":"2023-10-18T17:58:23.127657+0000","last_became_active":"2023-10-18T17:58:23.120522+0000","last_became_peered":"2023-10-18T17:58:23.120522+0000","last_unstale":"2023-10-18T17:58:23.127657+0000","last_undegraded":"2023-10-18T17:58:23.127657+0000","last_fullsized":"2023-10-18T17:58:23.127657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104694+0000","last_change":"2023-10-18T17:58:22.689656+0000","last_active":"2023-10-18T17:58:23.104694+0000","last_peered":"2023-10-18T17:58:23.104694+0000","last_clean":"2023-10-18T17:58:23.104694+0000","last_became_active":"2023-10-18T17:58:22.689376+0000","last_became_peered":"2023-10-18T17:58:22.689376+0000","last_unstale":"2023-10-18T17:58:23.104694+0000","last_undegraded":"2023-10-18T17:58:23.104694+0000","last_fullsized":"2023-10-18T17:58:23.104694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.126926+0000","last_change":"2023-10-18T17:58:23.120357+0000","last_active":"2023-10-18T17:58:23.126926+0000","last_peered":"2023-10-18T17:58:23.126926+0000","last_clean":"2023-10-18T17:58:23.126926+0000","last_became_active":"2023-10-18T17:58:23.118878+0000","last_became_peered":"2023-10-18T17:58:23.118878+0000","last_unstale":"2023-10-18T17:58:23.126926+0000","last_undegraded":"2023-10-18T17:58:23.126926+0000","last_fullsized":"2023-10-18T17:58:23.126926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128924+0000","last_change":"2023-10-18T17:58:23.126262+0000","last_active":"2023-10-18T17:58:23.128924+0000","last_peered":"2023-10-18T17:58:23.128924+0000","last_clean":"2023-10-18T17:58:23.128924+0000","last_became_active":"2023-10-18T17:58:23.125988+0000","last_became_peered":"2023-10-18T17:58:23.125988+0000","last_unstale":"2023-10-18T17:58:23.128924+0000","last_undegraded":"2023-10-18T17:58:23.128924+0000","last_fullsized":"2023-10-18T17:58:23.128924+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104422+0000","last_change":"2023-10-18T17:58:22.680967+0000","last_active":"2023-10-18T17:58:23.104422+0000","last_peered":"2023-10-18T17:58:23.104422+0000","last_clean":"2023-10-18T17:58:23.104422+0000","last_became_active":"2023-10-18T17:58:22.679976+0000","last_became_peered":"2023-10-18T17:58:22.679976+0000","last_unstale":"2023-10-18T17:58:23.104422+0000","last_undegraded":"2023-10-18T17:58:23.104422+0000","last_fullsized":"2023-10-18T17:58:23.104422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135616+0000","last_change":"2023-10-18T17:58:23.131014+0000","last_active":"2023-10-18T17:58:23.135616+0000","last_peered":"2023-10-18T17:58:23.135616+0000","last_clean":"2023-10-18T17:58:23.135616+0000","last_became_active":"2023-10-18T17:58:23.130516+0000","last_became_peered":"2023-10-18T17:58:23.130516+0000","last_unstale":"2023-10-18T17:58:23.135616+0000","last_undegraded":"2023-10-18T17:58:23.135616+0000","last_fullsized":"2023-10-18T17:58:23.135616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.109880+0000","last_change":"2023-10-18T17:58:22.684193+0000","last_active":"2023-10-18T17:58:23.109880+0000","last_peered":"2023-10-18T17:58:23.109880+0000","last_clean":"2023-10-18T17:58:23.109880+0000","last_became_active":"2023-10-18T17:58:22.683668+0000","last_became_peered":"2023-10-18T17:58:22.683668+0000","last_unstale":"2023-10-18T17:58:23.109880+0000","last_undegraded":"2023-10-18T17:58:23.109880+0000","last_fullsized":"2023-10-18T17:58:23.109880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.128357+0000","last_change":"2023-10-18T17:58:23.123700+0000","last_active":"2023-10-18T17:58:23.128357+0000","last_peered":"2023-10-18T17:58:23.128357+0000","last_clean":"2023-10-18T17:58:23.128357+0000","last_became_active":"2023-10-18T17:58:23.123224+0000","last_became_peered":"2023-10-18T17:58:23.123224+0000","last_unstale":"2023-10-18T17:58:23.128357+0000","last_undegraded":"2023-10-18T17:58:23.128357+0000","last_fullsized":"2023-10-18T17:58:23.128357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.116459+0000","last_change":"2023-10-18T17:58:22.674799+0000","last_active":"2023-10-18T17:58:23.116459+0000","last_peered":"2023-10-18T17:58:23.116459+0000","last_clean":"2023-10-18T17:58:23.116459+0000","last_became_active":"2023-10-18T17:58:22.674479+0000","last_became_peered":"2023-10-18T17:58:22.674479+0000","last_unstale":"2023-10-18T17:58:23.116459+0000","last_undegraded":"2023-10-18T17:58:23.116459+0000","last_fullsized":"2023-10-18T17:58:23.116459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.127727+0000","last_change":"2023-10-18T17:58:23.121289+0000","last_active":"2023-10-18T17:58:23.127727+0000","last_peered":"2023-10-18T17:58:23.127727+0000","last_clean":"2023-10-18T17:58:23.127727+0000","last_became_active":"2023-10-18T17:58:23.120024+0000","last_became_peered":"2023-10-18T17:58:23.120024+0000","last_unstale":"2023-10-18T17:58:23.127727+0000","last_undegraded":"2023-10-18T17:58:23.127727+0000","last_fullsized":"2023-10-18T17:58:23.127727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.110283+0000","last_change":"2023-10-18T17:58:22.675422+0000","last_active":"2023-10-18T17:58:23.110283+0000","last_peered":"2023-10-18T17:58:23.110283+0000","last_clean":"2023-10-18T17:58:23.110283+0000","last_became_active":"2023-10-18T17:58:22.675074+0000","last_became_peered":"2023-10-18T17:58:22.675074+0000","last_unstale":"2023-10-18T17:58:23.110283+0000","last_undegraded":"2023-10-18T17:58:23.110283+0000","last_fullsized":"2023-10-18T17:58:23.110283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.131984+0000","last_change":"2023-10-18T17:58:23.125653+0000","last_active":"2023-10-18T17:58:23.131984+0000","last_peered":"2023-10-18T17:58:23.131984+0000","last_clean":"2023-10-18T17:58:23.131984+0000","last_became_active":"2023-10-18T17:58:23.124682+0000","last_became_peered":"2023-10-18T17:58:23.124682+0000","last_unstale":"2023-10-18T17:58:23.131984+0000","last_undegraded":"2023-10-18T17:58:23.131984+0000","last_fullsized":"2023-10-18T17:58:23.131984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.115530+0000","last_change":"2023-10-18T17:58:22.673247+0000","last_active":"2023-10-18T17:58:23.115530+0000","last_peered":"2023-10-18T17:58:23.115530+0000","last_clean":"2023-10-18T17:58:23.115530+0000","last_became_active":"2023-10-18T17:58:22.672903+0000","last_became_peered":"2023-10-18T17:58:22.672903+0000","last_unstale":"2023-10-18T17:58:23.115530+0000","last_undegraded":"2023-10-18T17:58:23.115530+0000","last_fullsized":"2023-10-18T17:58:23.115530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129965+0000","last_change":"2023-10-18T17:58:23.125035+0000","last_active":"2023-10-18T17:58:23.129965+0000","last_peered":"2023-10-18T17:58:23.129965+0000","last_clean":"2023-10-18T17:58:23.129965+0000","last_became_active":"2023-10-18T17:58:23.124761+0000","last_became_peered":"2023-10-18T17:58:23.124761+0000","last_unstale":"2023-10-18T17:58:23.129965+0000","last_undegraded":"2023-10-18T17:58:23.129965+0000","last_fullsized":"2023-10-18T17:58:23.129965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.106275+0000","last_change":"2023-10-18T17:58:22.672712+0000","last_active":"2023-10-18T17:58:23.106275+0000","last_peered":"2023-10-18T17:58:23.106275+0000","last_clean":"2023-10-18T17:58:23.106275+0000","last_became_active":"2023-10-18T17:58:22.672368+0000","last_became_peered":"2023-10-18T17:58:22.672368+0000","last_unstale":"2023-10-18T17:58:23.106275+0000","last_undegraded":"2023-10-18T17:58:23.106275+0000","last_fullsized":"2023-10-18T17:58:23.106275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.135798+0000","last_change":"2023-10-18T17:58:23.127848+0000","last_active":"2023-10-18T17:58:23.135798+0000","last_peered":"2023-10-18T17:58:23.135798+0000","last_clean":"2023-10-18T17:58:23.135798+0000","last_became_active":"2023-10-18T17:58:23.125611+0000","last_became_peered":"2023-10-18T17:58:23.125611+0000","last_unstale":"2023-10-18T17:58:23.135798+0000","last_undegraded":"2023-10-18T17:58:23.135798+0000","last_fullsized":"2023-10-18T17:58:23.135798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.104555+0000","last_change":"2023-10-18T17:58:22.675355+0000","last_active":"2023-10-18T17:58:23.104555+0000","last_peered":"2023-10-18T17:58:23.104555+0000","last_clean":"2023-10-18T17:58:23.104555+0000","last_became_active":"2023-10-18T17:58:22.675050+0000","last_became_peered":"2023-10-18T17:58:22.675050+0000","last_unstale":"2023-10-18T17:58:23.104555+0000","last_undegraded":"2023-10-18T17:58:23.104555+0000","last_fullsized":"2023-10-18T17:58:23.104555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.129429+0000","last_change":"2023-10-18T17:58:23.124889+0000","last_active":"2023-10-18T17:58:23.129429+0000","last_peered":"2023-10-18T17:58:23.129429+0000","last_clean":"2023-10-18T17:58:23.129429+0000","last_became_active":"2023-10-18T17:58:23.124619+0000","last_became_peered":"2023-10-18T17:58:23.124619+0000","last_unstale":"2023-10-18T17:58:23.129429+0000","last_undegraded":"2023-10-18T17:58:23.129429+0000","last_fullsized":"2023-10-18T17:58:23.129429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:22.640242+0000","last_clean_scrub_stamp":"2023-10-18T17:58:22.640242+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-18T17:58:23.111694+0000","last_change":"2023-10-18T17:58:22.692437+0000","last_active":"2023-10-18T17:58:23.111694+0000","last_peered":"2023-10-18T17:58:23.111694+0000","last_clean":"2023-10-18T17:58:23.111694+0000","last_became_active":"2023-10-18T17:58:22.692043+0000","last_became_peered":"2023-10-18T17:58:22.692043+0000","last_unstale":"2023-10-18T17:58:23.111694+0000","last_undegraded":"2023-10-18T17:58:23.111694+0000","last_fullsized":"2023-10-18T17:58:23.111694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T17:58:21.630269+0000","last_clean_scrub_stamp":"2023-10-18T17:58:21.630269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297316,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074524,"statfs":{"total":96636764160,"available":96332312576,"internally_reserved":0,"allocated":225280,"data_stored":84635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T17:58:36.726 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-18T17:58:36.726 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-18T17:58:36.727 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-18T17:58:36.727 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:58:36.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c6c968700 1 Processor -- start 2023-10-18T17:58:36.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c6c968700 1 -- start start 2023-10-18T17:58:36.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c6c968700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f2c680658a0 0x7f2c68065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c6c968700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2c680c2bf0 0x7f2c680630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c6c968700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680be2e0 con 0x7f2c680635d0 2023-10-18T17:58:36.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c6c968700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c680bda80 con 0x7f2c680c2bf0 2023-10-18T17:58:36.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c6c968700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6806a7c0 con 0x7f2c680658a0 2023-10-18T17:58:36.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c5ffff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2c680c2bf0 0x7f2c680630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c5ffff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2c680c2bf0 0x7f2c680630e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:36566/0 (socket says 172.21.15.72:36566) 2023-10-18T17:58:36.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c5ffff700 1 -- 172.21.15.72:0/1710194380 learned_addr learned my addr 172.21.15.72:0/1710194380 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:36.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.914+0000 7f2c66d9d700 1 --2- 172.21.15.72:0/1710194380 >> v2:172.21.15.179:3300/0 conn(0x7f2c680658a0 0x7f2c68065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c5ffff700 1 -- 172.21.15.72:0/1710194380 >> v2:172.21.15.179:3300/0 conn(0x7f2c680658a0 msgr2=0x7f2c68065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:36.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c5ffff700 1 --2- 172.21.15.72:0/1710194380 >> v2:172.21.15.179:3300/0 conn(0x7f2c680658a0 0x7f2c68065c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c5ffff700 1 -- 172.21.15.72:0/1710194380 >> v1:172.21.15.72:6789/0 conn(0x7f2c680635d0 legacy=0x7f2c68151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c5ffff700 1 -- 172.21.15.72:0/1710194380 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680be770 con 0x7f2c680c2bf0 2023-10-18T17:58:36.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c66d9d700 1 --2- 172.21.15.72:0/1710194380 >> v2:172.21.15.179:3300/0 conn(0x7f2c680658a0 0x7f2c68065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:36.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6659c700 1 -- 172.21.15.72:0/1710194380 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566019161 0 0) 0x7f2c680be2e0 con 0x7f2c680635d0 2023-10-18T17:58:36.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c5ffff700 1 --2- 172.21.15.72:0/1710194380 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680c2bf0 0x7f2c680630e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2c50006fd0 tx=0x7f2c500090f0).ready entity=mon.0 client_cookie=a6cb441b3a9e3915 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:36.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6659c700 1 -- 172.21.15.72:0/1710194380 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5000b520 con 0x7f2c680c2bf0 2023-10-18T17:58:36.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6659c700 1 -- 172.21.15.72:0/1710194380 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5000b680 con 0x7f2c680c2bf0 2023-10-18T17:58:36.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6659c700 1 -- 172.21.15.72:0/1710194380 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5000ba00 con 0x7f2c680c2bf0 2023-10-18T17:58:36.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6c968700 1 -- 172.21.15.72:0/1710194380 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680c2bf0 msgr2=0x7f2c680630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6c968700 1 --2- 172.21.15.72:0/1710194380 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680c2bf0 0x7f2c680630e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2c50006fd0 tx=0x7f2c500090f0).stop 2023-10-18T17:58:36.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6c968700 1 -- 172.21.15.72:0/1710194380 shutdown_connections 2023-10-18T17:58:36.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6c968700 1 --2- 172.21.15.72:0/1710194380 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680c2bf0 0x7f2c680630e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6c968700 1 --2- 172.21.15.72:0/1710194380 >> v2:172.21.15.179:3300/0 conn(0x7f2c680658a0 0x7f2c68065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6c968700 1 -- 172.21.15.72:0/1710194380 >> 172.21.15.72:0/1710194380 conn(0x7f2c68062ba0 msgr2=0x7f2c680c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:36.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6c968700 1 -- 172.21.15.72:0/1710194380 shutdown_connections 2023-10-18T17:58:36.928 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.918+0000 7f2c6c968700 1 -- 172.21.15.72:0/1710194380 wait complete. 2023-10-18T17:58:36.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c6c968700 1 Processor -- start 2023-10-18T17:58:36.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c6c968700 1 -- start start 2023-10-18T17:58:36.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c6c968700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680635d0 0x7f2c68152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c6c968700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c680658a0 0x7f2c68151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c6c968700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c680c2bf0 0x7f2c68068ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c6c968700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680b7740 con 0x7f2c680635d0 2023-10-18T17:58:36.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c6c968700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68153260 con 0x7f2c680c2bf0 2023-10-18T17:58:36.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c6c968700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680b6b70 con 0x7f2c680658a0 2023-10-18T17:58:36.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c66d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680635d0 0x7f2c68152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c5ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c680658a0 0x7f2c68151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c5ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c680658a0 0x7f2c68151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:59442/0 (socket says 172.21.15.72:59442) 2023-10-18T17:58:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c5ffff700 1 -- 172.21.15.72:0/390745938 learned_addr learned my addr 172.21.15.72:0/390745938 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:36.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c6759e700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c680c2bf0 0x7f2c68068ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c66d9d700 1 -- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c680658a0 msgr2=0x7f2c68151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c66d9d700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c680658a0 0x7f2c68151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.933 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c66d9d700 1 -- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c680c2bf0 msgr2=0x7f2c68068ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:36.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c66d9d700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c680c2bf0 0x7f2c68068ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:36.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c66d9d700 1 -- 172.21.15.72:0/390745938 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c580092a0 con 0x7f2c680635d0 2023-10-18T17:58:36.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.922+0000 7f2c66d9d700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680635d0 0x7f2c68152c40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2c5800e980 tx=0x7f2c58009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:36.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.926+0000 7f2c5effd700 1 -- 172.21.15.72:0/390745938 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58012600 con 0x7f2c680635d0 2023-10-18T17:58:36.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.926+0000 7f2c5effd700 1 -- 172.21.15.72:0/390745938 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58012760 con 0x7f2c680635d0 2023-10-18T17:58:36.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.926+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c50009370 con 0x7f2c680635d0 2023-10-18T17:58:36.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.926+0000 7f2c5effd700 1 -- 172.21.15.72:0/390745938 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58012ae0 con 0x7f2c680635d0 2023-10-18T17:58:36.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.926+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c680b6e10 con 0x7f2c680635d0 2023-10-18T17:58:36.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.926+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c68046f00 con 0x7f2c680635d0 2023-10-18T17:58:36.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.926+0000 7f2c5effd700 1 -- 172.21.15.72:0/390745938 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2c5801b400 con 0x7f2c680635d0 2023-10-18T17:58:36.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.930+0000 7f2c5effd700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2c4805a540 0x7f2c4805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.930+0000 7f2c5effd700 1 -- 172.21.15.72:0/390745938 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2c58074c60 con 0x7f2c680635d0 2023-10-18T17:58:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.930+0000 7f2c5ffff700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2c4805a540 0x7f2c4805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.930+0000 7f2c5ffff700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2c4805a540 0x7f2c4805ca00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f2c680c2960 tx=0x7f2c50015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:36.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:36.934+0000 7f2c5effd700 1 -- 172.21.15.72:0/390745938 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f2c58048320 con 0x7f2c680635d0 2023-10-18T17:58:37.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.122+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2c680a8050 con 0x7f2c680635d0 2023-10-18T17:58:37.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.126+0000 7f2c5effd700 1 -- 172.21.15.72:0/390745938 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f2c58016020 con 0x7f2c680635d0 2023-10-18T17:58:37.129 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:37.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2c4805a540 msgr2=0x7f2c4805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:37.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2c4805a540 0x7f2c4805ca00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f2c680c2960 tx=0x7f2c50015000).stop 2023-10-18T17:58:37.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680635d0 msgr2=0x7f2c68152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:37.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680635d0 0x7f2c68152c40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2c5800e980 tx=0x7f2c58009060).stop 2023-10-18T17:58:37.134 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 shutdown_connections 2023-10-18T17:58:37.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2c4805a540 0x7f2c4805ca00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:37.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c680c2bf0 0x7f2c68068ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:37.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2c680658a0 0x7f2c68151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:37.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 --2- 172.21.15.72:0/390745938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2c680635d0 0x7f2c68152c40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:37.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 >> 172.21.15.72:0/390745938 conn(0x7f2c68062ba0 msgr2=0x7f2c680b5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:37.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 shutdown_connections 2023-10-18T17:58:37.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:37.130+0000 7f2c6c968700 1 -- 172.21.15.72:0/390745938 wait complete. 2023-10-18T17:58:37.157 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:58:38.448 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-10-18T17:58:40.158 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:58:40.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.310+0000 7f6abb7a9700 1 Processor -- start 2023-10-18T17:58:40.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.310+0000 7f6abb7a9700 1 -- start start 2023-10-18T17:58:40.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6abb7a9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6ab4067ff0 0x7f6ab40683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:40.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6abb7a9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40666b0 0x7f6ab40bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:40.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6abb7a9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab4065cd0 con 0x7f6ab40bde20 2023-10-18T17:58:40.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6abb7a9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab4066160 con 0x7f6ab4067ff0 2023-10-18T17:58:40.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6abb7a9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab4154750 con 0x7f6ab40666b0 2023-10-18T17:58:40.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab9545700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6ab4067ff0 0x7f6ab40683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:40.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab9545700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6ab4067ff0 0x7f6ab40683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46152/0 (socket says 172.21.15.72:46152) 2023-10-18T17:58:40.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab9545700 1 -- 172.21.15.72:0/1172251465 learned_addr learned my addr 172.21.15.72:0/1172251465 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:40.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab9d46700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f6ab40bde20 0x7f6ab4151410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:55056/0 (socket says 172.21.15.72:55056) 2023-10-18T17:58:40.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab8d44700 1 --2- 172.21.15.72:0/1172251465 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40666b0 0x7f6ab40bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:40.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab9545700 1 -- 172.21.15.72:0/1172251465 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40666b0 msgr2=0x7f6ab40bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:40.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab9545700 1 --2- 172.21.15.72:0/1172251465 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40666b0 0x7f6ab40bd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:40.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab9545700 1 -- 172.21.15.72:0/1172251465 >> v1:172.21.15.72:6789/0 conn(0x7f6ab40bde20 legacy=0x7f6ab4151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:40.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab9545700 1 -- 172.21.15.72:0/1172251465 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab4065100 con 0x7f6ab4067ff0 2023-10-18T17:58:40.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab8d44700 1 --2- 172.21.15.72:0/1172251465 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40666b0 0x7f6ab40bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:40.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6ab9545700 1 --2- 172.21.15.72:0/1172251465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab4067ff0 0x7f6ab40683d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f6ab0009f20 tx=0x7f6ab0009840).ready entity=mon.0 client_cookie=c1f9b5ba5a300d39 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:40.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6aabfff700 1 -- 172.21.15.72:0/1172251465 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000e070 con 0x7f6ab4067ff0 2023-10-18T17:58:40.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6aabfff700 1 -- 172.21.15.72:0/1172251465 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab000ba80 con 0x7f6ab4067ff0 2023-10-18T17:58:40.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6aabfff700 1 -- 172.21.15.72:0/1172251465 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000be00 con 0x7f6ab4067ff0 2023-10-18T17:58:40.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6abb7a9700 1 -- 172.21.15.72:0/1172251465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab4067ff0 msgr2=0x7f6ab40683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:40.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.314+0000 7f6abb7a9700 1 --2- 172.21.15.72:0/1172251465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab4067ff0 0x7f6ab40683d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f6ab0009f20 tx=0x7f6ab0009840).stop 2023-10-18T17:58:40.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.318+0000 7f6abb7a9700 1 -- 172.21.15.72:0/1172251465 shutdown_connections 2023-10-18T17:58:40.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.318+0000 7f6abb7a9700 1 --2- 172.21.15.72:0/1172251465 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40666b0 0x7f6ab40bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:40.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.318+0000 7f6abb7a9700 1 --2- 172.21.15.72:0/1172251465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab4067ff0 0x7f6ab40683d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:40.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.318+0000 7f6abb7a9700 1 -- 172.21.15.72:0/1172251465 >> 172.21.15.72:0/1172251465 conn(0x7f6ab40620d0 msgr2=0x7f6ab40624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:40.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.318+0000 7f6abb7a9700 1 -- 172.21.15.72:0/1172251465 shutdown_connections 2023-10-18T17:58:40.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.318+0000 7f6abb7a9700 1 -- 172.21.15.72:0/1172251465 wait complete. 2023-10-18T17:58:40.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.318+0000 7f6abb7a9700 1 Processor -- start 2023-10-18T17:58:40.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.318+0000 7f6abb7a9700 1 -- start start 2023-10-18T17:58:40.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6abb7a9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab40666b0 0x7f6ab4168520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:40.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6abb7a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab4067ff0 0x7f6ab40c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:40.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6abb7a9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40bde20 0x7f6ab40bbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:40.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6abb7a9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab4164580 con 0x7f6ab40666b0 2023-10-18T17:58:40.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6abb7a9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab40c2de0 con 0x7f6ab4067ff0 2023-10-18T17:58:40.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6abb7a9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab4168a10 con 0x7f6ab40bde20 2023-10-18T17:58:40.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9545700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab40666b0 0x7f6ab4168520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:40.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9545700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab40666b0 0x7f6ab4168520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46162/0 (socket says 172.21.15.72:46162) 2023-10-18T17:58:40.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9545700 1 -- 172.21.15.72:0/845494544 learned_addr learned my addr 172.21.15.72:0/845494544 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:40.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9d46700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40bde20 0x7f6ab40bbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:40.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab8d44700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab4067ff0 0x7f6ab40c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:40.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9545700 1 -- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40bde20 msgr2=0x7f6ab40bbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:40.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9545700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40bde20 0x7f6ab40bbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:40.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9545700 1 -- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab4067ff0 msgr2=0x7f6ab40c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:40.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9545700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab4067ff0 0x7f6ab40c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:40.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9545700 1 -- 172.21.15.72:0/845494544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab0009ae0 con 0x7f6ab40666b0 2023-10-18T17:58:40.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6ab9545700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab40666b0 0x7f6ab4168520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6ab0009360 tx=0x7f6ab0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:40.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6aa97fa700 1 -- 172.21.15.72:0/845494544 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0013a10 con 0x7f6ab40666b0 2023-10-18T17:58:40.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6aa97fa700 1 -- 172.21.15.72:0/845494544 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab0013b70 con 0x7f6ab40666b0 2023-10-18T17:58:40.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab4165360 con 0x7f6ab40666b0 2023-10-18T17:58:40.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6aa97fa700 1 -- 172.21.15.72:0/845494544 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab001c400 con 0x7f6ab40666b0 2023-10-18T17:58:40.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab4164790 con 0x7f6ab40666b0 2023-10-18T17:58:40.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.322+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab406c190 con 0x7f6ab40666b0 2023-10-18T17:58:40.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.326+0000 7f6aa97fa700 1 -- 172.21.15.72:0/845494544 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6ab001c7b0 con 0x7f6ab40666b0 2023-10-18T17:58:40.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.326+0000 7f6aa97fa700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6a9805a560 0x7f6a9805ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:40.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.326+0000 7f6aa97fa700 1 -- 172.21.15.72:0/845494544 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6ab0075e80 con 0x7f6ab40666b0 2023-10-18T17:58:40.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.326+0000 7f6ab8d44700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6a9805a560 0x7f6a9805ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:40.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.326+0000 7f6ab8d44700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6a9805a560 0x7f6a9805ca20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f6ab41552e0 tx=0x7f6aa4008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:40.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.334+0000 7f6aa97fa700 1 -- 172.21.15.72:0/845494544 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f6ab0049620 con 0x7f6ab40666b0 2023-10-18T17:58:40.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.526+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6ab4063cf0 con 0x7f6ab40666b0 2023-10-18T17:58:40.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.526+0000 7f6aa97fa700 1 -- 172.21.15.72:0/845494544 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f6ab004cc40 con 0x7f6ab40666b0 2023-10-18T17:58:40.532 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:40.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6a9805a560 msgr2=0x7f6a9805ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:40.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6a9805a560 0x7f6a9805ca20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f6ab41552e0 tx=0x7f6aa4008040).stop 2023-10-18T17:58:40.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab40666b0 msgr2=0x7f6ab4168520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:40.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab40666b0 0x7f6ab4168520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6ab0009360 tx=0x7f6ab0009840).stop 2023-10-18T17:58:40.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 shutdown_connections 2023-10-18T17:58:40.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6a9805a560 0x7f6a9805ca20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:40.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6ab40bde20 0x7f6ab40bbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:40.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab4067ff0 0x7f6ab40c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:40.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 --2- 172.21.15.72:0/845494544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ab40666b0 0x7f6ab4168520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:40.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 >> 172.21.15.72:0/845494544 conn(0x7f6ab40620d0 msgr2=0x7f6ab4156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:40.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 shutdown_connections 2023-10-18T17:58:40.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:40.530+0000 7f6abb7a9700 1 -- 172.21.15.72:0/845494544 wait complete. 2023-10-18T17:58:40.558 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:58:43.559 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:58:43.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.730+0000 7fb256a39700 1 Processor -- start 2023-10-18T17:58:43.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.730+0000 7fb256a39700 1 -- start start 2023-10-18T17:58:43.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb250065c80 0x7fb25014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:43.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25006a760 con 0x7fb250069f20 2023-10-18T17:58:43.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2500c2d40 con 0x7fb2500658a0 2023-10-18T17:58:43.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb250150bf0 con 0x7fb250065c80 2023-10-18T17:58:43.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb255236700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fb250069f20 0x7fb2500630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:55068/0 (socket says 172.21.15.72:55068) 2023-10-18T17:58:43.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb255236700 1 -- 172.21.15.72:0/1067567447 learned_addr learned my addr 172.21.15.72:0/1067567447 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:43.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256238700 1 --2- 172.21.15.72:0/1067567447 >> v2:172.21.15.72:3300/0 conn(0x7fb250065c80 0x7fb25014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:43.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb254a35700 1 -- 172.21.15.72:0/1067567447 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670412193 0 0) 0x7fb25006a760 con 0x7fb250069f20 2023-10-18T17:58:43.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb254a35700 1 -- 172.21.15.72:0/1067567447 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb23c0033f0 con 0x7fb250069f20 2023-10-18T17:58:43.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb254a35700 1 -- 172.21.15.72:0/1067567447 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403509321 0 0) 0x7fb2500c2d40 con 0x7fb2500658a0 2023-10-18T17:58:43.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb254a35700 1 -- 172.21.15.72:0/1067567447 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb25006a760 con 0x7fb2500658a0 2023-10-18T17:58:43.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256238700 1 -- 172.21.15.72:0/1067567447 >> v1:172.21.15.179:6789/0 conn(0x7fb2500658a0 legacy=0x7fb2500c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:43.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256238700 1 -- 172.21.15.72:0/1067567447 >> v1:172.21.15.72:6789/0 conn(0x7fb250069f20 legacy=0x7fb2500630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:43.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256238700 1 -- 172.21.15.72:0/1067567447 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25006a480 con 0x7fb250065c80 2023-10-18T17:58:43.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256238700 1 --2- 172.21.15.72:0/1067567447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 0x7fb25014cda0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb248009e60 tx=0x7fb248009780).ready entity=mon.0 client_cookie=32cc4e873b3ec113 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:43.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb254a35700 1 -- 172.21.15.72:0/1067567447 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24800e070 con 0x7fb250065c80 2023-10-18T17:58:43.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb254a35700 1 -- 172.21.15.72:0/1067567447 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24800bb40 con 0x7fb250065c80 2023-10-18T17:58:43.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb254a35700 1 -- 172.21.15.72:0/1067567447 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb248012410 con 0x7fb250065c80 2023-10-18T17:58:43.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 -- 172.21.15.72:0/1067567447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 msgr2=0x7fb25014cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:43.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 --2- 172.21.15.72:0/1067567447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 0x7fb25014cda0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb248009e60 tx=0x7fb248009780).stop 2023-10-18T17:58:43.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 -- 172.21.15.72:0/1067567447 shutdown_connections 2023-10-18T17:58:43.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 --2- 172.21.15.72:0/1067567447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 0x7fb25014cda0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:43.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 -- 172.21.15.72:0/1067567447 >> 172.21.15.72:0/1067567447 conn(0x7fb250062ba0 msgr2=0x7fb2500c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:43.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 -- 172.21.15.72:0/1067567447 shutdown_connections 2023-10-18T17:58:43.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.734+0000 7fb256a39700 1 -- 172.21.15.72:0/1067567447 wait complete. 2023-10-18T17:58:43.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb256a39700 1 Processor -- start 2023-10-18T17:58:43.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb256a39700 1 -- start start 2023-10-18T17:58:43.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb256a39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2500658a0 0x7fb2500ba090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:43.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb256a39700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 0x7fb2500b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:43.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb256a39700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb250069f20 0x7fb250162e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:43.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb256a39700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500bd290 con 0x7fb250065c80 2023-10-18T17:58:43.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb256a39700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2501639b0 con 0x7fb2500658a0 2023-10-18T17:58:43.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb256a39700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb250163c90 con 0x7fb250069f20 2023-10-18T17:58:43.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb255236700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 0x7fb2500b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:43.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb255236700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 0x7fb2500b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46172/0 (socket says 172.21.15.72:46172) 2023-10-18T17:58:43.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.738+0000 7fb255236700 1 -- 172.21.15.72:0/2050666842 learned_addr learned my addr 172.21.15.72:0/2050666842 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:43.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb256238700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb250069f20 0x7fb250162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:43.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb255a37700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2500658a0 0x7fb2500ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:43.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb255236700 1 -- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb250069f20 msgr2=0x7fb250162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:43.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb255236700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb250069f20 0x7fb250162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:43.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb255236700 1 -- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2500658a0 msgr2=0x7fb2500ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:43.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb255236700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2500658a0 0x7fb2500ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:43.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb255236700 1 -- 172.21.15.72:0/2050666842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb248009a20 con 0x7fb250065c80 2023-10-18T17:58:43.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb255a37700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2500658a0 0x7fb2500ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:43.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb256238700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb250069f20 0x7fb250162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:43.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb255236700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 0x7fb2500b50f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fb24000c490 tx=0x7fb240007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:43.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb245ffb700 1 -- 172.21.15.72:0/2050666842 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2400072e0 con 0x7fb250065c80 2023-10-18T17:58:43.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb245ffb700 1 -- 172.21.15.72:0/2050666842 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb240007440 con 0x7fb250065c80 2023-10-18T17:58:43.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2500bc960 con 0x7fb250065c80 2023-10-18T17:58:43.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb245ffb700 1 -- 172.21.15.72:0/2050666842 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2400057a0 con 0x7fb250065c80 2023-10-18T17:58:43.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2500bcf20 con 0x7fb250065c80 2023-10-18T17:58:43.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.742+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2500bd440 con 0x7fb250065c80 2023-10-18T17:58:43.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.746+0000 7fb245ffb700 1 -- 172.21.15.72:0/2050666842 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb240005b90 con 0x7fb250065c80 2023-10-18T17:58:43.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.746+0000 7fb245ffb700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb23405a510 0x7fb23405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:43.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.746+0000 7fb245ffb700 1 -- 172.21.15.72:0/2050666842 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb240073f90 con 0x7fb250065c80 2023-10-18T17:58:43.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.746+0000 7fb255a37700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb23405a510 0x7fb23405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:43.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.746+0000 7fb255a37700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb23405a510 0x7fb23405c9d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fb24c007fd0 tx=0x7fb24c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:43.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.754+0000 7fb245ffb700 1 -- 172.21.15.72:0/2050666842 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fb240047770 con 0x7fb250065c80 2023-10-18T17:58:43.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.950+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb250046f00 con 0x7fb250065c80 2023-10-18T17:58:43.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.950+0000 7fb245ffb700 1 -- 172.21.15.72:0/2050666842 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fb240018070 con 0x7fb250065c80 2023-10-18T17:58:43.956 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:43.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb23405a510 msgr2=0x7fb23405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:43.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb23405a510 0x7fb23405c9d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fb24c007fd0 tx=0x7fb24c009040).stop 2023-10-18T17:58:43.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 msgr2=0x7fb2500b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:43.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 0x7fb2500b50f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fb24000c490 tx=0x7fb240007d20).stop 2023-10-18T17:58:43.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 shutdown_connections 2023-10-18T17:58:43.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb23405a510 0x7fb23405c9d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:43.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb250069f20 0x7fb250162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:43.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb250065c80 0x7fb2500b50f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:43.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 --2- 172.21.15.72:0/2050666842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2500658a0 0x7fb2500ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:43.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 >> 172.21.15.72:0/2050666842 conn(0x7fb250062ba0 msgr2=0x7fb2500a5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:43.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 shutdown_connections 2023-10-18T17:58:43.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:43.954+0000 7fb256a39700 1 -- 172.21.15.72:0/2050666842 wait complete. 2023-10-18T17:58:43.977 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:58:45.262 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~14s 2023-10-18T17:58:46.978 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:58:47.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.146+0000 7ff07f307700 1 Processor -- start 2023-10-18T17:58:47.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.146+0000 7ff07f307700 1 -- start start 2023-10-18T17:58:47.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.150+0000 7ff07f307700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff078067d30 0x7ff0780630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:47.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.150+0000 7ff07f307700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff078068910 con 0x7ff078068180 2023-10-18T17:58:47.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.150+0000 7ff07f307700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff078150780 con 0x7ff07814d170 2023-10-18T17:58:47.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.150+0000 7ff07f307700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff078150c10 con 0x7ff078067d30 2023-10-18T17:58:47.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.150+0000 7ff07e305700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff07814d170 0x7ff07806a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:50666/0 (socket says 172.21.15.72:50666) 2023-10-18T17:58:47.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.150+0000 7ff07e305700 1 -- 172.21.15.72:0/1848490513 learned_addr learned my addr 172.21.15.72:0/1848490513 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:47.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.150+0000 7ff07db04700 1 --2- 172.21.15.72:0/1848490513 >> v2:172.21.15.179:3300/0 conn(0x7ff078067d30 0x7ff0780630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:47.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07d303700 1 -- 172.21.15.72:0/1848490513 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825527907 0 0) 0x7ff078150780 con 0x7ff07814d170 2023-10-18T17:58:47.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07d303700 1 -- 172.21.15.72:0/1848490513 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0640033f0 con 0x7ff07814d170 2023-10-18T17:58:47.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07db04700 1 -- 172.21.15.72:0/1848490513 >> v1:172.21.15.179:6789/0 conn(0x7ff07814d170 legacy=0x7ff07806a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:47.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07db04700 1 -- 172.21.15.72:0/1848490513 >> v1:172.21.15.72:6789/0 conn(0x7ff078068180 legacy=0x7ff078065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:47.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07db04700 1 -- 172.21.15.72:0/1848490513 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078068630 con 0x7ff078067d30 2023-10-18T17:58:47.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07db04700 1 --2- 172.21.15.72:0/1848490513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 0x7ff0780630e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff068009760 tx=0x7ff068009060).ready entity=mon.1 client_cookie=1429f9ec4f069a33 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:47.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07d303700 1 -- 172.21.15.72:0/1848490513 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06800b590 con 0x7ff078067d30 2023-10-18T17:58:47.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07d303700 1 -- 172.21.15.72:0/1848490513 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06800b6f0 con 0x7ff078067d30 2023-10-18T17:58:47.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07d303700 1 -- 172.21.15.72:0/1848490513 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06800ba70 con 0x7ff078067d30 2023-10-18T17:58:47.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07f307700 1 -- 172.21.15.72:0/1848490513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 msgr2=0x7ff0780630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:47.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07f307700 1 --2- 172.21.15.72:0/1848490513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 0x7ff0780630e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff068009760 tx=0x7ff068009060).stop 2023-10-18T17:58:47.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07f307700 1 -- 172.21.15.72:0/1848490513 shutdown_connections 2023-10-18T17:58:47.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07f307700 1 --2- 172.21.15.72:0/1848490513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 0x7ff0780630e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:47.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07f307700 1 -- 172.21.15.72:0/1848490513 >> 172.21.15.72:0/1848490513 conn(0x7ff078062ba0 msgr2=0x7ff0780c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:47.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07f307700 1 -- 172.21.15.72:0/1848490513 shutdown_connections 2023-10-18T17:58:47.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.154+0000 7ff07f307700 1 -- 172.21.15.72:0/1848490513 wait complete. 2023-10-18T17:58:47.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.158+0000 7ff07f307700 1 Processor -- start 2023-10-18T17:58:47.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.158+0000 7ff07f307700 1 -- start start 2023-10-18T17:58:47.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.158+0000 7ff07f307700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 0x7ff0780bed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:47.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.158+0000 7ff07f307700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff078068180 0x7ff0780bb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:47.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.158+0000 7ff07f307700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff07814d170 0x7ff0780bbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:47.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.158+0000 7ff07f307700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780c0700 con 0x7ff07814d170 2023-10-18T17:58:47.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.158+0000 7ff07f307700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780bd0d0 con 0x7ff078067d30 2023-10-18T17:58:47.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.158+0000 7ff07f307700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780bc150 con 0x7ff078068180 2023-10-18T17:58:47.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07e305700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 0x7ff0780bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:47.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07eb06700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff07814d170 0x7ff0780bbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:47.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07db04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff078068180 0x7ff0780bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:47.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07e305700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 0x7ff0780bed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:42274/0 (socket says 172.21.15.72:42274) 2023-10-18T17:58:47.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07e305700 1 -- 172.21.15.72:0/2873695441 learned_addr learned my addr 172.21.15.72:0/2873695441 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:47.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07e305700 1 -- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff078068180 msgr2=0x7ff0780bb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:47.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07e305700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff078068180 0x7ff0780bb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:47.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07e305700 1 -- 172.21.15.72:0/2873695441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff07814d170 msgr2=0x7ff0780bbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:47.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07e305700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff07814d170 0x7ff0780bbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:47.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07e305700 1 -- 172.21.15.72:0/2873695441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0680092a0 con 0x7ff078067d30 2023-10-18T17:58:47.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07e305700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 0x7ff0780bed90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7ff07400c420 tx=0x7ff074007c80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:47.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff06e7fc700 1 -- 172.21.15.72:0/2873695441 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07400ce60 con 0x7ff078067d30 2023-10-18T17:58:47.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff06e7fc700 1 -- 172.21.15.72:0/2873695441 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0740072e0 con 0x7ff078067d30 2023-10-18T17:58:47.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07814f2f0 con 0x7ff078067d30 2023-10-18T17:58:47.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff06e7fc700 1 -- 172.21.15.72:0/2873695441 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074007660 con 0x7ff078067d30 2023-10-18T17:58:47.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.162+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0780bdb40 con 0x7ff078067d30 2023-10-18T17:58:47.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.166+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0780bb0f0 con 0x7ff078067d30 2023-10-18T17:58:47.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.166+0000 7ff06e7fc700 1 -- 172.21.15.72:0/2873695441 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff0740059b0 con 0x7ff078067d30 2023-10-18T17:58:47.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.166+0000 7ff06e7fc700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff05c05a510 0x7ff05c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:47.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.170+0000 7ff06e7fc700 1 -- 172.21.15.72:0/2873695441 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff074073f90 con 0x7ff078067d30 2023-10-18T17:58:47.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.170+0000 7ff07db04700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff05c05a510 0x7ff05c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:47.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.170+0000 7ff07db04700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff05c05a510 0x7ff05c05c9d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7ff06800b350 tx=0x7ff068017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:47.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.178+0000 7ff06e7fc700 1 -- 172.21.15.72:0/2873695441 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ff0780bb0f0 con 0x7ff078067d30 2023-10-18T17:58:47.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.370+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff078046f00 con 0x7ff078067d30 2023-10-18T17:58:47.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff06e7fc700 1 -- 172.21.15.72:0/2873695441 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7ff0740476b0 con 0x7ff078067d30 2023-10-18T17:58:47.377 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:47.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff05c05a510 msgr2=0x7ff05c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:47.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff05c05a510 0x7ff05c05c9d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7ff06800b350 tx=0x7ff068017000).stop 2023-10-18T17:58:47.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 msgr2=0x7ff0780bed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:47.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 0x7ff0780bed90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7ff07400c420 tx=0x7ff074007c80).stop 2023-10-18T17:58:47.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 shutdown_connections 2023-10-18T17:58:47.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff05c05a510 0x7ff05c05c9d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:47.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff07814d170 0x7ff0780bbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:47.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff078068180 0x7ff0780bb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:47.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 --2- 172.21.15.72:0/2873695441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff078067d30 0x7ff0780bed90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:47.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 >> 172.21.15.72:0/2873695441 conn(0x7ff078062ba0 msgr2=0x7ff07814e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:47.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 shutdown_connections 2023-10-18T17:58:47.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:47.374+0000 7ff07f307700 1 -- 172.21.15.72:0/2873695441 wait complete. 2023-10-18T17:58:47.397 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:58:50.398 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:58:50.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc84d69700 1 Processor -- start 2023-10-18T17:58:50.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc84d69700 1 -- start start 2023-10-18T17:58:50.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc84d69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 0x7fbc800683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:50.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc84d69700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbc8014cfa0 0x7fbc8014d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:50.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc84d69700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc80065cf0 con 0x7fbc800666b0 2023-10-18T17:58:50.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc84d69700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc80150350 con 0x7fbc8014cfa0 2023-10-18T17:58:50.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc84d69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc800650e0 con 0x7fbc80067ff0 2023-10-18T17:58:50.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc7dd9b700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fbc800666b0 0x7fbc800c1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:52216/0 (socket says 172.21.15.72:52216) 2023-10-18T17:58:50.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc7dd9b700 1 -- 172.21.15.72:0/3363067055 learned_addr learned my addr 172.21.15.72:0/3363067055 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:50.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc7e59c700 1 --2- 172.21.15.72:0/3363067055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 0x7fbc800683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:50.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc7ed9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbc8014cfa0 0x7fbc8014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:50.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc7d59a700 1 -- 172.21.15.72:0/3363067055 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523391985 0 0) 0x7fbc80065cf0 con 0x7fbc800666b0 2023-10-18T17:58:50.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.565+0000 7fbc7d59a700 1 -- 172.21.15.72:0/3363067055 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc680033f0 con 0x7fbc800666b0 2023-10-18T17:58:50.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc7e59c700 1 -- 172.21.15.72:0/3363067055 >> v2:172.21.15.179:3300/0 conn(0x7fbc8014cfa0 msgr2=0x7fbc8014d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:58:50.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc7e59c700 1 --2- 172.21.15.72:0/3363067055 >> v2:172.21.15.179:3300/0 conn(0x7fbc8014cfa0 0x7fbc8014d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:50.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc7e59c700 1 -- 172.21.15.72:0/3363067055 >> v1:172.21.15.179:6789/0 conn(0x7fbc800666b0 legacy=0x7fbc800c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:50.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc7e59c700 1 -- 172.21.15.72:0/3363067055 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800653c0 con 0x7fbc80067ff0 2023-10-18T17:58:50.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc7ed9d700 1 --2- 172.21.15.72:0/3363067055 >> v2:172.21.15.179:3300/0 conn(0x7fbc8014cfa0 0x7fbc8014d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:50.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc7e59c700 1 --2- 172.21.15.72:0/3363067055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 0x7fbc800683d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbc70009760 tx=0x7fbc70009060).ready entity=mon.2 client_cookie=92ccbe8d34b39de1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:50.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc7d59a700 1 -- 172.21.15.72:0/3363067055 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000f510 con 0x7fbc80067ff0 2023-10-18T17:58:50.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc7d59a700 1 -- 172.21.15.72:0/3363067055 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7000f670 con 0x7fbc80067ff0 2023-10-18T17:58:50.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc7d59a700 1 -- 172.21.15.72:0/3363067055 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000f9f0 con 0x7fbc80067ff0 2023-10-18T17:58:50.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc84d69700 1 -- 172.21.15.72:0/3363067055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 msgr2=0x7fbc800683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:50.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc84d69700 1 --2- 172.21.15.72:0/3363067055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 0x7fbc800683d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbc70009760 tx=0x7fbc70009060).stop 2023-10-18T17:58:50.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc84d69700 1 -- 172.21.15.72:0/3363067055 shutdown_connections 2023-10-18T17:58:50.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc84d69700 1 --2- 172.21.15.72:0/3363067055 >> v2:172.21.15.179:3300/0 conn(0x7fbc8014cfa0 0x7fbc8014d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:50.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc84d69700 1 --2- 172.21.15.72:0/3363067055 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 0x7fbc800683d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:50.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc84d69700 1 -- 172.21.15.72:0/3363067055 >> 172.21.15.72:0/3363067055 conn(0x7fbc800620d0 msgr2=0x7fbc800624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:50.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc84d69700 1 -- 172.21.15.72:0/3363067055 shutdown_connections 2023-10-18T17:58:50.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.569+0000 7fbc84d69700 1 -- 172.21.15.72:0/3363067055 wait complete. 2023-10-18T17:58:50.580 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 Processor -- start 2023-10-18T17:58:50.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 -- start start 2023-10-18T17:58:50.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc800666b0 0x7fbc800b9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:50.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 0x7fbc800bcab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:50.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8014cfa0 0x7fbc801513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:50.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc80153ac0 con 0x7fbc800666b0 2023-10-18T17:58:50.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc801635c0 con 0x7fbc8014cfa0 2023-10-18T17:58:50.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc801638a0 con 0x7fbc80067ff0 2023-10-18T17:58:50.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7e59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc800666b0 0x7fbc800b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:50.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7e59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc800666b0 0x7fbc800b9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47850/0 (socket says 172.21.15.72:47850) 2023-10-18T17:58:50.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7e59c700 1 -- 172.21.15.72:0/3461601817 learned_addr learned my addr 172.21.15.72:0/3461601817 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:50.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7ed9d700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8014cfa0 0x7fbc801513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:50.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7dd9b700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 0x7fbc800bcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:50.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7e59c700 1 -- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 msgr2=0x7fbc800bcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:50.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7e59c700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 0x7fbc800bcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:50.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7e59c700 1 -- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8014cfa0 msgr2=0x7fbc801513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:50.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7e59c700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8014cfa0 0x7fbc801513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:50.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7e59c700 1 -- 172.21.15.72:0/3461601817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78009a20 con 0x7fbc800666b0 2023-10-18T17:58:50.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7ed9d700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8014cfa0 0x7fbc801513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:50.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc7e59c700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc800666b0 0x7fbc800b9570 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fbc70009d50 tx=0x7fbc70017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:50.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc6e7fc700 1 -- 172.21.15.72:0/3461601817 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000f9f0 con 0x7fbc800666b0 2023-10-18T17:58:50.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc6e7fc700 1 -- 172.21.15.72:0/3461601817 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc70000c90 con 0x7fbc800666b0 2023-10-18T17:58:50.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc700092a0 con 0x7fbc800666b0 2023-10-18T17:58:50.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc6e7fc700 1 -- 172.21.15.72:0/3461601817 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7001b660 con 0x7fbc800666b0 2023-10-18T17:58:50.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.573+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc80163e60 con 0x7fbc800666b0 2023-10-18T17:58:50.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.577+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8006c190 con 0x7fbc800666b0 2023-10-18T17:58:50.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.577+0000 7fbc6e7fc700 1 -- 172.21.15.72:0/3461601817 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbc7001b9d0 con 0x7fbc800666b0 2023-10-18T17:58:50.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.577+0000 7fbc6e7fc700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbc6005a540 0x7fbc6005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:50.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.581+0000 7fbc6e7fc700 1 -- 172.21.15.72:0/3461601817 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbc70076340 con 0x7fbc800666b0 2023-10-18T17:58:50.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.581+0000 7fbc7dd9b700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbc6005a540 0x7fbc6005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:50.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.581+0000 7fbc7dd9b700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbc6005a540 0x7fbc6005ca00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fbc80067b40 tx=0x7fbc74009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:50.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.585+0000 7fbc6e7fc700 1 -- 172.21.15.72:0/3461601817 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fbc70049a60 con 0x7fbc800666b0 2023-10-18T17:58:50.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.777+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbc800656a0 con 0x7fbc800666b0 2023-10-18T17:58:50.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.777+0000 7fbc6e7fc700 1 -- 172.21.15.72:0/3461601817 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fbc7004d080 con 0x7fbc800666b0 2023-10-18T17:58:50.783 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbc6005a540 msgr2=0x7fbc6005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbc6005a540 0x7fbc6005ca00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fbc80067b40 tx=0x7fbc74009040).stop 2023-10-18T17:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc800666b0 msgr2=0x7fbc800b9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:50.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc800666b0 0x7fbc800b9570 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fbc70009d50 tx=0x7fbc70017000).stop 2023-10-18T17:58:50.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 shutdown_connections 2023-10-18T17:58:50.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbc6005a540 0x7fbc6005ca00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:50.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc8014cfa0 0x7fbc801513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:50.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbc80067ff0 0x7fbc800bcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:50.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 --2- 172.21.15.72:0/3461601817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc800666b0 0x7fbc800b9570 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:50.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 >> 172.21.15.72:0/3461601817 conn(0x7fbc800620d0 msgr2=0x7fbc8014bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:50.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 shutdown_connections 2023-10-18T17:58:50.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:50.781+0000 7fbc84d69700 1 -- 172.21.15.72:0/3461601817 wait complete. 2023-10-18T17:58:50.804 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:58:52.073 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2023-10-18T17:58:53.806 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:58:53.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.961+0000 7fda541a9700 1 Processor -- start 2023-10-18T17:58:53.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.961+0000 7fda541a9700 1 -- start start 2023-10-18T17:58:53.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda541a9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fda4c14cfa0 0x7fda4c14d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:53.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda541a9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c065cf0 con 0x7fda4c067ff0 2023-10-18T17:58:53.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda541a9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c150350 con 0x7fda4c0666b0 2023-10-18T17:58:53.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda541a9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4c0650e0 con 0x7fda4c14cfa0 2023-10-18T17:58:53.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda52746700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fda4c14cfa0 0x7fda4c14d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:53.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda51f45700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fda4c067ff0 0x7fda4c0683d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56254/0 (socket says 172.21.15.72:56254) 2023-10-18T17:58:53.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda51f45700 1 -- 172.21.15.72:0/2819696361 learned_addr learned my addr 172.21.15.72:0/2819696361 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:53.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda50f43700 1 -- 172.21.15.72:0/2819696361 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393044712 0 0) 0x7fda4c150350 con 0x7fda4c0666b0 2023-10-18T17:58:53.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda50f43700 1 -- 172.21.15.72:0/2819696361 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda380033f0 con 0x7fda4c0666b0 2023-10-18T17:58:53.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda50f43700 1 -- 172.21.15.72:0/2819696361 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131137034 0 0) 0x7fda4c065cf0 con 0x7fda4c067ff0 2023-10-18T17:58:53.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda52746700 1 -- 172.21.15.72:0/2819696361 >> v1:172.21.15.179:6789/0 conn(0x7fda4c0666b0 legacy=0x7fda4c0c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:53.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda52746700 1 -- 172.21.15.72:0/2819696361 >> v1:172.21.15.72:6789/0 conn(0x7fda4c067ff0 legacy=0x7fda4c0683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:53.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda52746700 1 -- 172.21.15.72:0/2819696361 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c0653c0 con 0x7fda4c14cfa0 2023-10-18T17:58:53.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda52746700 1 --2- 172.21.15.72:0/2819696361 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c14cfa0 0x7fda4c14d430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fda44009e60 tx=0x7fda44009780).ready entity=mon.0 client_cookie=1b3c325fad2e0d57 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:53.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda50f43700 1 -- 172.21.15.72:0/2819696361 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4400e070 con 0x7fda4c14cfa0 2023-10-18T17:58:53.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda50f43700 1 -- 172.21.15.72:0/2819696361 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4400bbf0 con 0x7fda4c14cfa0 2023-10-18T17:58:53.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda50f43700 1 -- 172.21.15.72:0/2819696361 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44012410 con 0x7fda4c14cfa0 2023-10-18T17:58:53.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda541a9700 1 -- 172.21.15.72:0/2819696361 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c14cfa0 msgr2=0x7fda4c14d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:53.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.965+0000 7fda541a9700 1 --2- 172.21.15.72:0/2819696361 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c14cfa0 0x7fda4c14d430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fda44009e60 tx=0x7fda44009780).stop 2023-10-18T17:58:53.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 -- 172.21.15.72:0/2819696361 shutdown_connections 2023-10-18T17:58:53.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 --2- 172.21.15.72:0/2819696361 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c14cfa0 0x7fda4c14d430 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:53.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 -- 172.21.15.72:0/2819696361 >> 172.21.15.72:0/2819696361 conn(0x7fda4c0620d0 msgr2=0x7fda4c0624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:53.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 -- 172.21.15.72:0/2819696361 shutdown_connections 2023-10-18T17:58:53.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 -- 172.21.15.72:0/2819696361 wait complete. 2023-10-18T17:58:53.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 Processor -- start 2023-10-18T17:58:53.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 -- start start 2023-10-18T17:58:53.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c0666b0 0x7fda4c152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:53.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda4c067ff0 0x7fda4c148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:53.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda4c14cfa0 0x7fda4c149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:53.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c149780 con 0x7fda4c0666b0 2023-10-18T17:58:53.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0bffc0 con 0x7fda4c14cfa0 2023-10-18T17:58:53.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda541a9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0bf6e0 con 0x7fda4c067ff0 2023-10-18T17:58:53.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.969+0000 7fda51f45700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c0666b0 0x7fda4c152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:53.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda51f45700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c0666b0 0x7fda4c152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47876/0 (socket says 172.21.15.72:47876) 2023-10-18T17:58:53.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda51f45700 1 -- 172.21.15.72:0/3794803788 learned_addr learned my addr 172.21.15.72:0/3794803788 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:53.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda52746700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda4c14cfa0 0x7fda4c149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:53.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda51744700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda4c067ff0 0x7fda4c148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:53.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda51f45700 1 -- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda4c067ff0 msgr2=0x7fda4c148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:53.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda51f45700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda4c067ff0 0x7fda4c148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:53.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda51f45700 1 -- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda4c14cfa0 msgr2=0x7fda4c149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:53.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda51f45700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda4c14cfa0 0x7fda4c149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:53.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda51f45700 1 -- 172.21.15.72:0/3794803788 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda44009a20 con 0x7fda4c0666b0 2023-10-18T17:58:53.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda52746700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda4c14cfa0 0x7fda4c149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:58:53.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda51f45700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c0666b0 0x7fda4c152790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fda4800c510 tx=0x7fda48007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:53.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda41ffb700 1 -- 172.21.15.72:0/3794803788 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4800ce80 con 0x7fda4c0666b0 2023-10-18T17:58:53.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda41ffb700 1 -- 172.21.15.72:0/3794803788 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda480072e0 con 0x7fda4c0666b0 2023-10-18T17:58:53.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c0ba2a0 con 0x7fda4c0666b0 2023-10-18T17:58:53.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda41ffb700 1 -- 172.21.15.72:0/3794803788 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48007660 con 0x7fda4c0666b0 2023-10-18T17:58:53.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c0bf970 con 0x7fda4c0666b0 2023-10-18T17:58:53.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.973+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4c06c190 con 0x7fda4c0666b0 2023-10-18T17:58:53.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.977+0000 7fda41ffb700 1 -- 172.21.15.72:0/3794803788 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fda480059b0 con 0x7fda4c0666b0 2023-10-18T17:58:53.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.977+0000 7fda41ffb700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fda3005a510 0x7fda3005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:53.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.977+0000 7fda41ffb700 1 -- 172.21.15.72:0/3794803788 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fda48073f60 con 0x7fda4c0666b0 2023-10-18T17:58:53.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.977+0000 7fda51744700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fda3005a510 0x7fda3005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:53.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.977+0000 7fda51744700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fda3005a510 0x7fda3005c9d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fda4c0c3980 tx=0x7fda3c007bb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:53.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:53.981+0000 7fda41ffb700 1 -- 172.21.15.72:0/3794803788 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fda48047740 con 0x7fda4c0666b0 2023-10-18T17:58:54.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.177+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fda4c0656a0 con 0x7fda4c0666b0 2023-10-18T17:58:54.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.177+0000 7fda41ffb700 1 -- 172.21.15.72:0/3794803788 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fda48018070 con 0x7fda4c0666b0 2023-10-18T17:58:54.182 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fda3005a510 msgr2=0x7fda3005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:54.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fda3005a510 0x7fda3005c9d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fda4c0c3980 tx=0x7fda3c007bb0).stop 2023-10-18T17:58:54.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c0666b0 msgr2=0x7fda4c152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:54.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c0666b0 0x7fda4c152790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fda4800c510 tx=0x7fda48007c70).stop 2023-10-18T17:58:54.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 shutdown_connections 2023-10-18T17:58:54.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fda3005a510 0x7fda3005c9d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:54.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fda4c14cfa0 0x7fda4c149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fda4c067ff0 0x7fda4c148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 --2- 172.21.15.72:0/3794803788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fda4c0666b0 0x7fda4c152790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 >> 172.21.15.72:0/3794803788 conn(0x7fda4c0620d0 msgr2=0x7fda4c0c23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 shutdown_connections 2023-10-18T17:58:54.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:54.181+0000 7fda541a9700 1 -- 172.21.15.72:0/3794803788 wait complete. 2023-10-18T17:58:54.204 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:58:56.403 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:58:56.411 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:58:57.205 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:58:57.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.365+0000 7fe08a0db700 1 Processor -- start 2023-10-18T17:58:57.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.365+0000 7fe08a0db700 1 -- start start 2023-10-18T17:58:57.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.365+0000 7fe08a0db700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe084067ff0 0x7fe0840683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:57.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.365+0000 7fe08a0db700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe084065cd0 con 0x7fe0840666b0 2023-10-18T17:58:57.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.365+0000 7fe08a0db700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe084154750 con 0x7fe084151410 2023-10-18T17:58:57.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.365+0000 7fe08a0db700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe084066160 con 0x7fe084067ff0 2023-10-18T17:58:57.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.365+0000 7fe07bfff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fe0840666b0 0x7fe0840bd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56264/0 (socket says 172.21.15.72:56264) 2023-10-18T17:58:57.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.365+0000 7fe0837fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe084067ff0 0x7fe0840683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:57.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.365+0000 7fe07bfff700 1 -- 172.21.15.72:0/2455669321 learned_addr learned my addr 172.21.15.72:0/2455669321 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:58:57.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe082ffd700 1 -- 172.21.15.72:0/2455669321 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109297692 0 0) 0x7fe084065cd0 con 0x7fe0840666b0 2023-10-18T17:58:57.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe082ffd700 1 -- 172.21.15.72:0/2455669321 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0700033f0 con 0x7fe0840666b0 2023-10-18T17:58:57.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe082ffd700 1 -- 172.21.15.72:0/2455669321 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774829896 0 0) 0x7fe084154750 con 0x7fe084151410 2023-10-18T17:58:57.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe082ffd700 1 -- 172.21.15.72:0/2455669321 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe084065cd0 con 0x7fe084151410 2023-10-18T17:58:57.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe082ffd700 1 -- 172.21.15.72:0/2455669321 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3408550371 0 0) 0x7fe0700033f0 con 0x7fe0840666b0 2023-10-18T17:58:57.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe0837fe700 1 -- 172.21.15.72:0/2455669321 >> v1:172.21.15.179:6789/0 conn(0x7fe084151410 legacy=0x7fe084151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:57.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe0837fe700 1 -- 172.21.15.72:0/2455669321 >> v1:172.21.15.72:6789/0 conn(0x7fe0840666b0 legacy=0x7fe0840bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:57.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe0837fe700 1 -- 172.21.15.72:0/2455669321 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe084065100 con 0x7fe084067ff0 2023-10-18T17:58:57.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe0837fe700 1 --2- 172.21.15.72:0/2455669321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe084067ff0 0x7fe0840683d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fe074009760 tx=0x7fe074009060).ready entity=mon.0 client_cookie=3a993e205b17564d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:57.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe082ffd700 1 -- 172.21.15.72:0/2455669321 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c001470 con 0x7fe084067ff0 2023-10-18T17:58:57.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe082ffd700 1 -- 172.21.15.72:0/2455669321 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe074011400 con 0x7fe084067ff0 2023-10-18T17:58:57.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe082ffd700 1 -- 172.21.15.72:0/2455669321 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074011780 con 0x7fe084067ff0 2023-10-18T17:58:57.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe08a0db700 1 -- 172.21.15.72:0/2455669321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe084067ff0 msgr2=0x7fe0840683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:57.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe08a0db700 1 --2- 172.21.15.72:0/2455669321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe084067ff0 0x7fe0840683d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fe074009760 tx=0x7fe074009060).stop 2023-10-18T17:58:57.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe08a0db700 1 -- 172.21.15.72:0/2455669321 shutdown_connections 2023-10-18T17:58:57.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe08a0db700 1 --2- 172.21.15.72:0/2455669321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe084067ff0 0x7fe0840683d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:57.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe08a0db700 1 -- 172.21.15.72:0/2455669321 >> 172.21.15.72:0/2455669321 conn(0x7fe0840620d0 msgr2=0x7fe0840624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:57.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe08a0db700 1 -- 172.21.15.72:0/2455669321 shutdown_connections 2023-10-18T17:58:57.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.369+0000 7fe08a0db700 1 -- 172.21.15.72:0/2455669321 wait complete. 2023-10-18T17:58:57.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.373+0000 7fe08a0db700 1 Processor -- start 2023-10-18T17:58:57.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.373+0000 7fe08a0db700 1 -- start start 2023-10-18T17:58:57.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe08a0db700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0840666b0 0x7fe084164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:57.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe08a0db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe084067ff0 0x7fe084168370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:57.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe08a0db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe084151410 0x7fe084153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:57.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe08a0db700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840c0590 con 0x7fe0840666b0 2023-10-18T17:58:57.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe08a0db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084152a20 con 0x7fe084067ff0 2023-10-18T17:58:57.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe08a0db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840bffd0 con 0x7fe084151410 2023-10-18T17:58:57.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe0837fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0840666b0 0x7fe084164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:57.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe0837fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0840666b0 0x7fe084164eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47896/0 (socket says 172.21.15.72:47896) 2023-10-18T17:58:57.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe0837fe700 1 -- 172.21.15.72:0/2481371893 learned_addr learned my addr 172.21.15.72:0/2481371893 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:58:57.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe083fff700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe084151410 0x7fe084153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:57.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe07bfff700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe084067ff0 0x7fe084168370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:57.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe0837fe700 1 -- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe084151410 msgr2=0x7fe084153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:57.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe0837fe700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe084151410 0x7fe084153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:57.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe0837fe700 1 -- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe084067ff0 msgr2=0x7fe084168370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:57.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe0837fe700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe084067ff0 0x7fe084168370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe0837fe700 1 -- 172.21.15.72:0/2481371893 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0740092a0 con 0x7fe0840666b0 2023-10-18T17:58:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe07bfff700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe084067ff0 0x7fe084168370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:58:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe0837fe700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0840666b0 0x7fe084164eb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fe0740118a0 tx=0x7fe074016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe07b7fe700 1 -- 172.21.15.72:0/2481371893 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0740108a0 con 0x7fe0840666b0 2023-10-18T17:58:57.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe07b7fe700 1 -- 172.21.15.72:0/2481371893 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe074010a00 con 0x7fe0840666b0 2023-10-18T17:58:57.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0840c02b0 con 0x7fe0840666b0 2023-10-18T17:58:57.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe07b7fe700 1 -- 172.21.15.72:0/2481371893 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074010d80 con 0x7fe0840666b0 2023-10-18T17:58:57.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe084164990 con 0x7fe0840666b0 2023-10-18T17:58:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.377+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe084164bb0 con 0x7fe0840666b0 2023-10-18T17:58:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.381+0000 7fe07b7fe700 1 -- 172.21.15.72:0/2481371893 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe074018070 con 0x7fe0840666b0 2023-10-18T17:58:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.381+0000 7fe07b7fe700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe06405a510 0x7fe06405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:58:57.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.381+0000 7fe07b7fe700 1 -- 172.21.15.72:0/2481371893 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe074075ff0 con 0x7fe0840666b0 2023-10-18T17:58:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.381+0000 7fe07bfff700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe06405a510 0x7fe06405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:58:57.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.381+0000 7fe07bfff700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe06405a510 0x7fe06405c9d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fe084158fa0 tx=0x7fe06c0015e0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:58:57.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.385+0000 7fe07b7fe700 1 -- 172.21.15.72:0/2481371893 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fe0740a4ac0 con 0x7fe0840666b0 2023-10-18T17:58:57.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.577+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe08406c190 con 0x7fe0840666b0 2023-10-18T17:58:57.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.577+0000 7fe07b7fe700 1 -- 172.21.15.72:0/2481371893 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fe074049790 con 0x7fe0840666b0 2023-10-18T17:58:57.583 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:58:57.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe06405a510 msgr2=0x7fe06405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:57.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe06405a510 0x7fe06405c9d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fe084158fa0 tx=0x7fe06c0015e0).stop 2023-10-18T17:58:57.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0840666b0 msgr2=0x7fe084164eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:58:57.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0840666b0 0x7fe084164eb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fe0740118a0 tx=0x7fe074016000).stop 2023-10-18T17:58:57.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 shutdown_connections 2023-10-18T17:58:57.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe06405a510 0x7fe06405c9d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:57.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe084151410 0x7fe084153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:57.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe084067ff0 0x7fe084168370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:57.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 --2- 172.21.15.72:0/2481371893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0840666b0 0x7fe084164eb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:58:57.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 >> 172.21.15.72:0/2481371893 conn(0x7fe0840620d0 msgr2=0x7fe0840624b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:58:57.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 shutdown_connections 2023-10-18T17:58:57.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:58:57.581+0000 7fe08a0db700 1 -- 172.21.15.72:0/2481371893 wait complete. 2023-10-18T17:58:57.608 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:58:58.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~27s 2023-10-18T17:59:00.610 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:00.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57f701b700 1 Processor -- start 2023-10-18T17:59:00.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57f701b700 1 -- start start 2023-10-18T17:59:00.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57f701b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f57f00bca50 0x7f57f00b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:00.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57f701b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f00bc480 con 0x7f57f00b9690 2023-10-18T17:59:00.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57f701b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f0065190 con 0x7f57f0065d30 2023-10-18T17:59:00.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57f701b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f00bb920 con 0x7f57f00bca50 2023-10-18T17:59:00.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57f4db7700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f57f00b9690 0x7f57f0065620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:49724/0 (socket says 172.21.15.72:49724) 2023-10-18T17:59:00.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57f4db7700 1 -- 172.21.15.72:0/1886441259 learned_addr learned my addr 172.21.15.72:0/1886441259 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:00.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57effff700 1 --2- 172.21.15.72:0/1886441259 >> v2:172.21.15.179:3300/0 conn(0x7f57f00bca50 0x7f57f00b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:00.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57ef7fe700 1 -- 172.21.15.72:0/1886441259 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86740789 0 0) 0x7f57f00bc480 con 0x7f57f00b9690 2023-10-18T17:59:00.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57ef7fe700 1 -- 172.21.15.72:0/1886441259 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e00033f0 con 0x7f57f00b9690 2023-10-18T17:59:00.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57ef7fe700 1 -- 172.21.15.72:0/1886441259 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282466639 0 0) 0x7f57f0065190 con 0x7f57f0065d30 2023-10-18T17:59:00.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.769+0000 7f57ef7fe700 1 -- 172.21.15.72:0/1886441259 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57f00bc480 con 0x7f57f0065d30 2023-10-18T17:59:00.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57effff700 1 -- 172.21.15.72:0/1886441259 >> v1:172.21.15.179:6789/0 conn(0x7f57f0065d30 legacy=0x7f57f00b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:00.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57effff700 1 -- 172.21.15.72:0/1886441259 >> v1:172.21.15.72:6789/0 conn(0x7f57f00b9690 legacy=0x7f57f0065620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:00.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57effff700 1 -- 172.21.15.72:0/1886441259 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00bbbe0 con 0x7f57f00bca50 2023-10-18T17:59:00.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57effff700 1 --2- 172.21.15.72:0/1886441259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f00bca50 0x7f57f00b9a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f57dc009760 tx=0x7f57dc009060).ready entity=mon.1 client_cookie=2ae1bdd9e9681e4d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:00.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57ef7fe700 1 -- 172.21.15.72:0/1886441259 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00d4a0 con 0x7f57f00bca50 2023-10-18T17:59:00.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57ef7fe700 1 -- 172.21.15.72:0/1886441259 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc00d600 con 0x7f57f00bca50 2023-10-18T17:59:00.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57ef7fe700 1 -- 172.21.15.72:0/1886441259 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00d980 con 0x7f57f00bca50 2023-10-18T17:59:00.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57f701b700 1 -- 172.21.15.72:0/1886441259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f00bca50 msgr2=0x7f57f00b9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:00.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57f701b700 1 --2- 172.21.15.72:0/1886441259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f00bca50 0x7f57f00b9a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f57dc009760 tx=0x7f57dc009060).stop 2023-10-18T17:59:00.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57f701b700 1 -- 172.21.15.72:0/1886441259 shutdown_connections 2023-10-18T17:59:00.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57f701b700 1 --2- 172.21.15.72:0/1886441259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f00bca50 0x7f57f00b9a70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:00.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57f701b700 1 -- 172.21.15.72:0/1886441259 >> 172.21.15.72:0/1886441259 conn(0x7f57f00620d0 msgr2=0x7f57f00624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:00.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57f701b700 1 -- 172.21.15.72:0/1886441259 shutdown_connections 2023-10-18T17:59:00.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57f701b700 1 -- 172.21.15.72:0/1886441259 wait complete. 2023-10-18T17:59:00.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.773+0000 7f57f701b700 1 Processor -- start 2023-10-18T17:59:00.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f701b700 1 -- start start 2023-10-18T17:59:00.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f701b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f0065d30 0x7f57f00c1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:00.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f701b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f00b9690 0x7f57f0165100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:00.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f701b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f0148d00 0x7f57f0149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:00.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f701b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f014de70 con 0x7f57f00b9690 2023-10-18T17:59:00.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f701b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00c0b70 con 0x7f57f0065d30 2023-10-18T17:59:00.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f701b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0162140 con 0x7f57f0148d00 2023-10-18T17:59:00.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57effff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f00b9690 0x7f57f0165100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:00.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f4db7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f0065d30 0x7f57f00c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:00.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57effff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f00b9690 0x7f57f0165100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57914/0 (socket says 172.21.15.72:57914) 2023-10-18T17:59:00.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57effff700 1 -- 172.21.15.72:0/4288612005 learned_addr learned my addr 172.21.15.72:0/4288612005 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:00.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f55b8700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f0148d00 0x7f57f0149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:00.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57effff700 1 -- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f0148d00 msgr2=0x7f57f0149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:00.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57effff700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f0148d00 0x7f57f0149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:00.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57effff700 1 -- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f0065d30 msgr2=0x7f57f00c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:00.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57effff700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f0065d30 0x7f57f00c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:00.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57effff700 1 -- 172.21.15.72:0/4288612005 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc0092a0 con 0x7f57f00b9690 2023-10-18T17:59:00.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f4db7700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f0065d30 0x7f57f00c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:59:00.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57effff700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f00b9690 0x7f57f0165100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f57dc010000 tx=0x7f57dc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:00.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57ecff9700 1 -- 172.21.15.72:0/4288612005 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00dcf0 con 0x7f57f00b9690 2023-10-18T17:59:00.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57ecff9700 1 -- 172.21.15.72:0/4288612005 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc00de50 con 0x7f57f00b9690 2023-10-18T17:59:00.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f01655f0 con 0x7f57f00b9690 2023-10-18T17:59:00.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.777+0000 7f57ecff9700 1 -- 172.21.15.72:0/4288612005 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00d980 con 0x7f57f00b9690 2023-10-18T17:59:00.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.781+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f00bff60 con 0x7f57f00b9690 2023-10-18T17:59:00.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.781+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f00c0ce0 con 0x7f57f00b9690 2023-10-18T17:59:00.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.781+0000 7f57ecff9700 1 -- 172.21.15.72:0/4288612005 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f57dc01d640 con 0x7f57f00b9690 2023-10-18T17:59:00.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.781+0000 7f57ecff9700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f57d405a510 0x7f57d405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:00.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.781+0000 7f57ecff9700 1 -- 172.21.15.72:0/4288612005 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f57dc076110 con 0x7f57f00b9690 2023-10-18T17:59:00.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.781+0000 7f57f4db7700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f57d405a510 0x7f57d405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:00.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.785+0000 7f57f4db7700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f57d405a510 0x7f57d405c9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f57e4005ad0 tx=0x7f57e4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:00.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.789+0000 7f57ecff9700 1 -- 172.21.15.72:0/4288612005 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f57dc049850 con 0x7f57f00b9690 2023-10-18T17:59:00.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.981+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f57f006c190 con 0x7f57f00b9690 2023-10-18T17:59:00.987 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.981+0000 7f57ecff9700 1 -- 172.21.15.72:0/4288612005 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f57dc04ce70 con 0x7f57f00b9690 2023-10-18T17:59:00.987 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:00.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f57d405a510 msgr2=0x7f57d405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:00.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f57d405a510 0x7f57d405c9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f57e4005ad0 tx=0x7f57e4009040).stop 2023-10-18T17:59:00.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f00b9690 msgr2=0x7f57f0165100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:00.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f00b9690 0x7f57f0165100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f57dc010000 tx=0x7f57dc016000).stop 2023-10-18T17:59:00.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 shutdown_connections 2023-10-18T17:59:00.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f57d405a510 0x7f57d405c9d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:00.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f57f0148d00 0x7f57f0149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:00.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f57f00b9690 0x7f57f0165100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:00.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 --2- 172.21.15.72:0/4288612005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57f0065d30 0x7f57f00c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:00.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 >> 172.21.15.72:0/4288612005 conn(0x7f57f00620d0 msgr2=0x7f57f014acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:00.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 shutdown_connections 2023-10-18T17:59:00.994 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:00.985+0000 7f57f701b700 1 -- 172.21.15.72:0/4288612005 wait complete. 2023-10-18T17:59:01.009 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:04.011 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:04.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.173+0000 7fb8088b9700 1 Processor -- start 2023-10-18T17:59:04.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.173+0000 7fb8088b9700 1 -- start start 2023-10-18T17:59:04.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.173+0000 7fb8088b9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb804063350 0x7fb8040637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:04.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.173+0000 7fb8088b9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80414c380 con 0x7fb8040bc930 2023-10-18T17:59:04.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.173+0000 7fb8088b9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80414c640 con 0x7fb8041498d0 2023-10-18T17:59:04.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.173+0000 7fb8088b9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80414c920 con 0x7fb804063350 2023-10-18T17:59:04.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb803fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb804063350 0x7fb8040637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:04.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb8037fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fb8041498d0 0x7fb8040bd370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:58380/0 (socket says 172.21.15.72:58380) 2023-10-18T17:59:04.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb8037fe700 1 -- 172.21.15.72:0/302329024 learned_addr learned my addr 172.21.15.72:0/302329024 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:04.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb802ffd700 1 -- 172.21.15.72:0/302329024 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757485181 0 0) 0x7fb80414c640 con 0x7fb8041498d0 2023-10-18T17:59:04.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb802ffd700 1 -- 172.21.15.72:0/302329024 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f00033f0 con 0x7fb8041498d0 2023-10-18T17:59:04.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb803fff700 1 -- 172.21.15.72:0/302329024 >> v1:172.21.15.179:6789/0 conn(0x7fb8041498d0 legacy=0x7fb8040bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:04.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb803fff700 1 -- 172.21.15.72:0/302329024 >> v1:172.21.15.72:6789/0 conn(0x7fb8040bc930 legacy=0x7fb8040bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:04.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb803fff700 1 -- 172.21.15.72:0/302329024 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80414cc00 con 0x7fb804063350 2023-10-18T17:59:04.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb803fff700 1 --2- 172.21.15.72:0/302329024 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb804063350 0x7fb8040637c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb7fc00afa0 tx=0x7fb7fc00a890).ready entity=mon.2 client_cookie=8cdc25b1deef577c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:04.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb802ffd700 1 -- 172.21.15.72:0/302329024 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc00f070 con 0x7fb804063350 2023-10-18T17:59:04.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb802ffd700 1 -- 172.21.15.72:0/302329024 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc00a2e0 con 0x7fb804063350 2023-10-18T17:59:04.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb802ffd700 1 -- 172.21.15.72:0/302329024 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc00cc60 con 0x7fb804063350 2023-10-18T17:59:04.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb8088b9700 1 -- 172.21.15.72:0/302329024 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb804063350 msgr2=0x7fb8040637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:04.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb8088b9700 1 --2- 172.21.15.72:0/302329024 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb804063350 0x7fb8040637c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb7fc00afa0 tx=0x7fb7fc00a890).stop 2023-10-18T17:59:04.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb8088b9700 1 -- 172.21.15.72:0/302329024 shutdown_connections 2023-10-18T17:59:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb8088b9700 1 --2- 172.21.15.72:0/302329024 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb804063350 0x7fb8040637c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb8088b9700 1 -- 172.21.15.72:0/302329024 >> 172.21.15.72:0/302329024 conn(0x7fb804062ba0 msgr2=0x7fb8040c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb8088b9700 1 -- 172.21.15.72:0/302329024 shutdown_connections 2023-10-18T17:59:04.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.177+0000 7fb8088b9700 1 -- 172.21.15.72:0/302329024 wait complete. 2023-10-18T17:59:04.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.181+0000 7fb8088b9700 1 Processor -- start 2023-10-18T17:59:04.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.181+0000 7fb8088b9700 1 -- start start 2023-10-18T17:59:04.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8088b9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb804063350 0x7fb8040b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:04.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8088b9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8040bc930 0x7fb804160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:04.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8088b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8041498d0 0x7fb804148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:04.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8088b9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8041610e0 con 0x7fb8041498d0 2023-10-18T17:59:04.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8088b9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040b64b0 con 0x7fb804063350 2023-10-18T17:59:04.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8088b9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040ba800 con 0x7fb8040bc930 2023-10-18T17:59:04.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb803fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8041498d0 0x7fb804148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:04.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb803fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8041498d0 0x7fb804148610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57930/0 (socket says 172.21.15.72:57930) 2023-10-18T17:59:04.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb803fff700 1 -- 172.21.15.72:0/2217012108 learned_addr learned my addr 172.21.15.72:0/2217012108 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:04.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8037fe700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb804063350 0x7fb8040b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:04.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb7faffd700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8040bc930 0x7fb804160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:04.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb803fff700 1 -- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8040bc930 msgr2=0x7fb804160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:04.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb803fff700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8040bc930 0x7fb804160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:04.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb803fff700 1 -- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb804063350 msgr2=0x7fb8040b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:04.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb803fff700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb804063350 0x7fb8040b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:04.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb803fff700 1 -- 172.21.15.72:0/2217012108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc00ab30 con 0x7fb8041498d0 2023-10-18T17:59:04.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8037fe700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb804063350 0x7fb8040b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T17:59:04.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb803fff700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8041498d0 0x7fb804148610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fb7fc00c810 tx=0x7fb7fc00a890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:04.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb7fbfff700 1 -- 172.21.15.72:0/2217012108 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc009ba0 con 0x7fb8041498d0 2023-10-18T17:59:04.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb7fbfff700 1 -- 172.21.15.72:0/2217012108 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc009d00 con 0x7fb8041498d0 2023-10-18T17:59:04.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8040b9710 con 0x7fb8041498d0 2023-10-18T17:59:04.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb7fbfff700 1 -- 172.21.15.72:0/2217012108 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc014670 con 0x7fb8041498d0 2023-10-18T17:59:04.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.185+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8040b9cd0 con 0x7fb8041498d0 2023-10-18T17:59:04.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.189+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8040b6660 con 0x7fb8041498d0 2023-10-18T17:59:04.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.189+0000 7fb7fbfff700 1 -- 172.21.15.72:0/2217012108 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb7fc003040 con 0x7fb8041498d0 2023-10-18T17:59:04.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.193+0000 7fb7fbfff700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb7e405a510 0x7fb7e405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:04.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.193+0000 7fb7fbfff700 1 -- 172.21.15.72:0/2217012108 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb7fc0758d0 con 0x7fb8041498d0 2023-10-18T17:59:04.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.193+0000 7fb8037fe700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb7e405a510 0x7fb7e405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:04.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.193+0000 7fb8037fe700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb7e405a510 0x7fb7e405c9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fb7f4007fd0 tx=0x7fb7f4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:04.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.197+0000 7fb7fbfff700 1 -- 172.21.15.72:0/2217012108 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fb8040b6660 con 0x7fb8041498d0 2023-10-18T17:59:04.398 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.393+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb804046f00 con 0x7fb8041498d0 2023-10-18T17:59:04.399 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.393+0000 7fb7fbfff700 1 -- 172.21.15.72:0/2217012108 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fb7fc022080 con 0x7fb8041498d0 2023-10-18T17:59:04.399 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:04.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb7e405a510 msgr2=0x7fb7e405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:04.401 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb7e405a510 0x7fb7e405c9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fb7f4007fd0 tx=0x7fb7f4009040).stop 2023-10-18T17:59:04.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8041498d0 msgr2=0x7fb804148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:04.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8041498d0 0x7fb804148610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fb7fc00c810 tx=0x7fb7fc00a890).stop 2023-10-18T17:59:04.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 shutdown_connections 2023-10-18T17:59:04.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb7e405a510 0x7fb7e405c9d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:04.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb8041498d0 0x7fb804148610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:04.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb8040bc930 0x7fb804160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:04.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 --2- 172.21.15.72:0/2217012108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb804063350 0x7fb8040b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:04.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 >> 172.21.15.72:0/2217012108 conn(0x7fb804062ba0 msgr2=0x7fb8040bdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:04.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 shutdown_connections 2023-10-18T17:59:04.404 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:04.397+0000 7fb8088b9700 1 -- 172.21.15.72:0/2217012108 wait complete. 2023-10-18T17:59:04.421 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:05.704 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~34s 2023-10-18T17:59:07.423 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:07.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.581+0000 7fe36359e700 1 Processor -- start 2023-10-18T17:59:07.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.581+0000 7fe36359e700 1 -- start start 2023-10-18T17:59:07.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36359e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe3640658a0 0x7fe364065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:07.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36359e700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36406a760 con 0x7fe364069f20 2023-10-18T17:59:07.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36359e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3640c2d40 con 0x7fe3640630e0 2023-10-18T17:59:07.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36359e700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe364150bf0 con 0x7fe3640658a0 2023-10-18T17:59:07.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36259c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fe3640658a0 0x7fe364065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:07.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe361d9b700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fe364069f20 0x7fe3640c2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:49750/0 (socket says 172.21.15.72:49750) 2023-10-18T17:59:07.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe361d9b700 1 -- 172.21.15.72:0/3906618076 learned_addr learned my addr 172.21.15.72:0/3906618076 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:07.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36159a700 1 -- 172.21.15.72:0/3906618076 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606079392 0 0) 0x7fe3640c2d40 con 0x7fe3640630e0 2023-10-18T17:59:07.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36159a700 1 -- 172.21.15.72:0/3906618076 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe34c0033f0 con 0x7fe3640630e0 2023-10-18T17:59:07.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36159a700 1 -- 172.21.15.72:0/3906618076 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363099924 0 0) 0x7fe36406a760 con 0x7fe364069f20 2023-10-18T17:59:07.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36159a700 1 -- 172.21.15.72:0/3906618076 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3640c2d40 con 0x7fe364069f20 2023-10-18T17:59:07.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36259c700 1 -- 172.21.15.72:0/3906618076 >> v1:172.21.15.179:6789/0 conn(0x7fe3640630e0 legacy=0x7fe36414cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:07.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36259c700 1 -- 172.21.15.72:0/3906618076 >> v1:172.21.15.72:6789/0 conn(0x7fe364069f20 legacy=0x7fe3640c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:07.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36259c700 1 -- 172.21.15.72:0/3906618076 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36406a480 con 0x7fe3640658a0 2023-10-18T17:59:07.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36259c700 1 --2- 172.21.15.72:0/3906618076 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3640658a0 0x7fe364065c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fe354009760 tx=0x7fe354009060).ready entity=mon.0 client_cookie=51916ad5fdb5be20 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:07.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36159a700 1 -- 172.21.15.72:0/3906618076 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35400f510 con 0x7fe3640658a0 2023-10-18T17:59:07.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36159a700 1 -- 172.21.15.72:0/3906618076 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35400f670 con 0x7fe3640658a0 2023-10-18T17:59:07.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36159a700 1 -- 172.21.15.72:0/3906618076 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35400f9f0 con 0x7fe3640658a0 2023-10-18T17:59:07.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36359e700 1 -- 172.21.15.72:0/3906618076 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3640658a0 msgr2=0x7fe364065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:07.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.585+0000 7fe36359e700 1 --2- 172.21.15.72:0/3906618076 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3640658a0 0x7fe364065c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fe354009760 tx=0x7fe354009060).stop 2023-10-18T17:59:07.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.589+0000 7fe36359e700 1 -- 172.21.15.72:0/3906618076 shutdown_connections 2023-10-18T17:59:07.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.589+0000 7fe36359e700 1 --2- 172.21.15.72:0/3906618076 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3640658a0 0x7fe364065c80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:07.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.589+0000 7fe36359e700 1 -- 172.21.15.72:0/3906618076 >> 172.21.15.72:0/3906618076 conn(0x7fe364062ba0 msgr2=0x7fe3640c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:07.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.589+0000 7fe36359e700 1 -- 172.21.15.72:0/3906618076 shutdown_connections 2023-10-18T17:59:07.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.589+0000 7fe36359e700 1 -- 172.21.15.72:0/3906618076 wait complete. 2023-10-18T17:59:07.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.589+0000 7fe36359e700 1 Processor -- start 2023-10-18T17:59:07.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.589+0000 7fe36359e700 1 -- start start 2023-10-18T17:59:07.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe36359e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3640630e0 0x7fe3640b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:07.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe36359e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3640658a0 0x7fe3640b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:07.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe36359e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe364069f20 0x7fe364156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:07.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe36359e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640b8430 con 0x7fe364069f20 2023-10-18T17:59:07.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe36359e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe364160c30 con 0x7fe3640630e0 2023-10-18T17:59:07.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe36359e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640b7820 con 0x7fe3640658a0 2023-10-18T17:59:07.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe362d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe364069f20 0x7fe364156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:07.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe362d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe364069f20 0x7fe364156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57948/0 (socket says 172.21.15.72:57948) 2023-10-18T17:59:07.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe362d9d700 1 -- 172.21.15.72:0/3644842956 learned_addr learned my addr 172.21.15.72:0/3644842956 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:07.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe361d9b700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3640658a0 0x7fe3640b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:07.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe36259c700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3640630e0 0x7fe3640b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:07.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe362d9d700 1 -- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3640658a0 msgr2=0x7fe3640b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:07.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe362d9d700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3640658a0 0x7fe3640b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:07.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe362d9d700 1 -- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3640630e0 msgr2=0x7fe3640b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:07.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe362d9d700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3640630e0 0x7fe3640b6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:07.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe362d9d700 1 -- 172.21.15.72:0/3644842956 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3540092a0 con 0x7fe364069f20 2023-10-18T17:59:07.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe362d9d700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe364069f20 0x7fe364156b20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fe35c00c7c0 tx=0x7fe35c002660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:07.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe3527fc700 1 -- 172.21.15.72:0/3644842956 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c010510 con 0x7fe364069f20 2023-10-18T17:59:07.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe3527fc700 1 -- 172.21.15.72:0/3644842956 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c010670 con 0x7fe364069f20 2023-10-18T17:59:07.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe3527fc700 1 -- 172.21.15.72:0/3644842956 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c010a30 con 0x7fe364069f20 2023-10-18T17:59:07.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe364157f20 con 0x7fe364069f20 2023-10-18T17:59:07.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.593+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe364160300 con 0x7fe364069f20 2023-10-18T17:59:07.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.597+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36414fd90 con 0x7fe364069f20 2023-10-18T17:59:07.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.597+0000 7fe3527fc700 1 -- 172.21.15.72:0/3644842956 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fe35c01b400 con 0x7fe364069f20 2023-10-18T17:59:07.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.597+0000 7fe3527fc700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe34405a540 0x7fe34405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:07.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.597+0000 7fe3527fc700 1 -- 172.21.15.72:0/3644842956 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe35c074600 con 0x7fe364069f20 2023-10-18T17:59:07.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.601+0000 7fe36259c700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe34405a540 0x7fe34405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:07.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.601+0000 7fe36259c700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe34405a540 0x7fe34405ca00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe354006e00 tx=0x7fe354017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:07.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.601+0000 7fe3527fc700 1 -- 172.21.15.72:0/3644842956 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fe35c047cc0 con 0x7fe364069f20 2023-10-18T17:59:07.805 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.801+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe364046f00 con 0x7fe364069f20 2023-10-18T17:59:07.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.801+0000 7fe3527fc700 1 -- 172.21.15.72:0/3644842956 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fe35c04b2e0 con 0x7fe364069f20 2023-10-18T17:59:07.806 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:07.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe34405a540 msgr2=0x7fe34405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:07.809 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe34405a540 0x7fe34405ca00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fe354006e00 tx=0x7fe354017000).stop 2023-10-18T17:59:07.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe364069f20 msgr2=0x7fe364156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:07.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe364069f20 0x7fe364156b20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fe35c00c7c0 tx=0x7fe35c002660).stop 2023-10-18T17:59:07.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 shutdown_connections 2023-10-18T17:59:07.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fe34405a540 0x7fe34405ca00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:07.810 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe364069f20 0x7fe364156b20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:07.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fe3640658a0 0x7fe3640b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:07.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 --2- 172.21.15.72:0/3644842956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3640630e0 0x7fe3640b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:07.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 >> 172.21.15.72:0/3644842956 conn(0x7fe364062ba0 msgr2=0x7fe3640a60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:07.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 shutdown_connections 2023-10-18T17:59:07.811 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:07.805+0000 7fe36359e700 1 -- 172.21.15.72:0/3644842956 wait complete. 2023-10-18T17:59:07.827 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:09.548 INFO:tasks.ceph.mds.f.smithi179.stderr:terminate called after throwing an instance of 'boost::wrapexcept' 2023-10-18T17:59:09.548 INFO:tasks.ceph.mds.f.smithi179.stderr: what(): boost::bad_get: failed value get using boost::get 2023-10-18T17:59:09.548 INFO:tasks.ceph.mds.f.smithi179.stderr:*** Caught signal (Aborted) ** 2023-10-18T17:59:09.549 INFO:tasks.ceph.mds.f.smithi179.stderr: in thread 7f3379816700 thread_name:ms_dispatch 2023-10-18T17:59:09.549 INFO:tasks.ceph.mds.f.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:09.549 INFO:tasks.ceph.mds.f.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f337ea75420] 2023-10-18T17:59:09.550 INFO:tasks.ceph.mds.f.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:09.550 INFO:tasks.ceph.mds.f.smithi179.stderr: 3: abort() 2023-10-18T17:59:09.550 INFO:tasks.ceph.mds.f.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f337e917911] 2023-10-18T17:59:09.550 INFO:tasks.ceph.mds.f.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f337e92338c] 2023-10-18T17:59:09.550 INFO:tasks.ceph.mds.f.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f337e9233f7] 2023-10-18T17:59:09.551 INFO:tasks.ceph.mds.f.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f337e9236a9] 2023-10-18T17:59:09.551 INFO:tasks.ceph.mds.f.smithi179.stderr: 8: ceph-mds(+0x185107) [0x55fc65179107] 2023-10-18T17:59:09.551 INFO:tasks.ceph.mds.f.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x55fc6517972d] 2023-10-18T17:59:09.551 INFO:tasks.ceph.mds.f.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55fc651eb09f] 2023-10-18T17:59:09.551 INFO:tasks.ceph.mds.f.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55fc651eca95] 2023-10-18T17:59:09.552 INFO:tasks.ceph.mds.f.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55fc651b7b10] 2023-10-18T17:59:09.552 INFO:tasks.ceph.mds.f.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55fc651bbabf] 2023-10-18T17:59:09.552 INFO:tasks.ceph.mds.f.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55fc651bc456] 2023-10-18T17:59:09.552 INFO:tasks.ceph.mds.f.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f337ef7b710] 2023-10-18T17:59:09.552 INFO:tasks.ceph.mds.f.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f337ef78faf] 2023-10-18T17:59:09.553 INFO:tasks.ceph.mds.f.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f337f047b21] 2023-10-18T17:59:09.553 INFO:tasks.ceph.mds.f.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f337ea69609] 2023-10-18T17:59:09.553 INFO:tasks.ceph.mds.f.smithi179.stderr: 19: clone() 2023-10-18T17:59:09.554 INFO:tasks.ceph.mds.f.smithi179.stderr:2023-10-18T17:59:09.544+0000 7f3379816700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:09.554 INFO:tasks.ceph.mds.f.smithi179.stderr: in thread 7f3379816700 thread_name:ms_dispatch 2023-10-18T17:59:09.554 INFO:tasks.ceph.mds.f.smithi179.stderr: 2023-10-18T17:59:09.554 INFO:tasks.ceph.mds.f.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:09.555 INFO:tasks.ceph.mds.f.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f337ea75420] 2023-10-18T17:59:09.555 INFO:tasks.ceph.mds.f.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:09.555 INFO:tasks.ceph.mds.f.smithi179.stderr: 3: abort() 2023-10-18T17:59:09.555 INFO:tasks.ceph.mds.f.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f337e917911] 2023-10-18T17:59:09.555 INFO:tasks.ceph.mds.f.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f337e92338c] 2023-10-18T17:59:09.556 INFO:tasks.ceph.mds.f.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f337e9233f7] 2023-10-18T17:59:09.556 INFO:tasks.ceph.mds.f.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f337e9236a9] 2023-10-18T17:59:09.556 INFO:tasks.ceph.mds.f.smithi179.stderr: 8: ceph-mds(+0x185107) [0x55fc65179107] 2023-10-18T17:59:09.556 INFO:tasks.ceph.mds.f.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x55fc6517972d] 2023-10-18T17:59:09.556 INFO:tasks.ceph.mds.f.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55fc651eb09f] 2023-10-18T17:59:09.556 INFO:tasks.ceph.mds.f.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55fc651eca95] 2023-10-18T17:59:09.557 INFO:tasks.ceph.mds.f.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55fc651b7b10] 2023-10-18T17:59:09.557 INFO:tasks.ceph.mds.f.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55fc651bbabf] 2023-10-18T17:59:09.557 INFO:tasks.ceph.mds.f.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55fc651bc456] 2023-10-18T17:59:09.557 INFO:tasks.ceph.mds.f.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f337ef7b710] 2023-10-18T17:59:09.557 INFO:tasks.ceph.mds.f.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f337ef78faf] 2023-10-18T17:59:09.557 INFO:tasks.ceph.mds.f.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f337f047b21] 2023-10-18T17:59:09.558 INFO:tasks.ceph.mds.f.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f337ea69609] 2023-10-18T17:59:09.558 INFO:tasks.ceph.mds.f.smithi179.stderr: 19: clone() 2023-10-18T17:59:09.558 INFO:tasks.ceph.mds.f.smithi179.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:09.558 INFO:tasks.ceph.mds.f.smithi179.stderr: 2023-10-18T17:59:09.558 INFO:tasks.ceph.mds.f.smithi179.stderr: 0> 2023-10-18T17:59:09.544+0000 7f3379816700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:09.559 INFO:tasks.ceph.mds.f.smithi179.stderr: in thread 7f3379816700 thread_name:ms_dispatch 2023-10-18T17:59:09.559 INFO:tasks.ceph.mds.f.smithi179.stderr: 2023-10-18T17:59:09.559 INFO:tasks.ceph.mds.f.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:09.559 INFO:tasks.ceph.mds.f.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f337ea75420] 2023-10-18T17:59:09.559 INFO:tasks.ceph.mds.f.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:09.559 INFO:tasks.ceph.mds.f.smithi179.stderr: 3: abort() 2023-10-18T17:59:09.560 INFO:tasks.ceph.mds.f.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f337e917911] 2023-10-18T17:59:09.560 INFO:tasks.ceph.mds.f.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f337e92338c] 2023-10-18T17:59:09.560 INFO:tasks.ceph.mds.f.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f337e9233f7] 2023-10-18T17:59:09.560 INFO:tasks.ceph.mds.f.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f337e9236a9] 2023-10-18T17:59:09.560 INFO:tasks.ceph.mds.f.smithi179.stderr: 8: ceph-mds(+0x185107) [0x55fc65179107] 2023-10-18T17:59:09.561 INFO:tasks.ceph.mds.f.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x55fc6517972d] 2023-10-18T17:59:09.561 INFO:tasks.ceph.mds.f.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55fc651eb09f] 2023-10-18T17:59:09.561 INFO:tasks.ceph.mds.f.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55fc651eca95] 2023-10-18T17:59:09.561 INFO:tasks.ceph.mds.f.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55fc651b7b10] 2023-10-18T17:59:09.561 INFO:tasks.ceph.mds.f.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55fc651bbabf] 2023-10-18T17:59:09.562 INFO:tasks.ceph.mds.f.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55fc651bc456] 2023-10-18T17:59:09.562 INFO:tasks.ceph.mds.f.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f337ef7b710] 2023-10-18T17:59:09.562 INFO:tasks.ceph.mds.f.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f337ef78faf] 2023-10-18T17:59:09.562 INFO:tasks.ceph.mds.f.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f337f047b21] 2023-10-18T17:59:09.562 INFO:tasks.ceph.mds.f.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f337ea69609] 2023-10-18T17:59:09.563 INFO:tasks.ceph.mds.f.smithi179.stderr: 19: clone() 2023-10-18T17:59:09.563 INFO:tasks.ceph.mds.f.smithi179.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:09.563 INFO:tasks.ceph.mds.f.smithi179.stderr: 2023-10-18T17:59:09.564 INFO:tasks.ceph.mds.f.smithi179.stderr: 0> 2023-10-18T17:59:09.544+0000 7f3379816700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:09.564 INFO:tasks.ceph.mds.f.smithi179.stderr: in thread 7f3379816700 thread_name:ms_dispatch 2023-10-18T17:59:09.564 INFO:tasks.ceph.mds.f.smithi179.stderr: 2023-10-18T17:59:09.564 INFO:tasks.ceph.mds.f.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:09.564 INFO:tasks.ceph.mds.f.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f337ea75420] 2023-10-18T17:59:09.565 INFO:tasks.ceph.mds.f.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:09.565 INFO:tasks.ceph.mds.f.smithi179.stderr: 3: abort() 2023-10-18T17:59:09.565 INFO:tasks.ceph.mds.f.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f337e917911] 2023-10-18T17:59:09.565 INFO:tasks.ceph.mds.f.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f337e92338c] 2023-10-18T17:59:09.566 INFO:tasks.ceph.mds.f.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f337e9233f7] 2023-10-18T17:59:09.566 INFO:tasks.ceph.mds.f.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f337e9236a9] 2023-10-18T17:59:09.566 INFO:tasks.ceph.mds.f.smithi179.stderr: 8: ceph-mds(+0x185107) [0x55fc65179107] 2023-10-18T17:59:09.566 INFO:tasks.ceph.mds.f.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x55fc6517972d] 2023-10-18T17:59:09.567 INFO:tasks.ceph.mds.f.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55fc651eb09f] 2023-10-18T17:59:09.567 INFO:tasks.ceph.mds.f.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55fc651eca95] 2023-10-18T17:59:09.567 INFO:tasks.ceph.mds.f.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55fc651b7b10] 2023-10-18T17:59:09.567 INFO:tasks.ceph.mds.f.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55fc651bbabf] 2023-10-18T17:59:09.568 INFO:tasks.ceph.mds.f.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55fc651bc456] 2023-10-18T17:59:09.568 INFO:tasks.ceph.mds.f.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f337ef7b710] 2023-10-18T17:59:09.568 INFO:tasks.ceph.mds.f.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f337ef78faf] 2023-10-18T17:59:09.568 INFO:tasks.ceph.mds.f.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f337f047b21] 2023-10-18T17:59:09.568 INFO:tasks.ceph.mds.f.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f337ea69609] 2023-10-18T17:59:09.569 INFO:tasks.ceph.mds.f.smithi179.stderr: 19: clone() 2023-10-18T17:59:09.569 INFO:tasks.ceph.mds.f.smithi179.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:09.569 INFO:tasks.ceph.mds.f.smithi179.stderr: 2023-10-18T17:59:09.773 INFO:tasks.ceph.mds.f.smithi179.stderr:daemon-helper: command crashed with signal 6 2023-10-18T17:59:10.828 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:11.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:10.997+0000 7fd8b7769700 1 Processor -- start 2023-10-18T17:59:11.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:10.997+0000 7fd8b7769700 1 -- start start 2023-10-18T17:59:11.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b7769700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b0067ff0 0x7fd8b00683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:11.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b7769700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd8b00666b0 0x7fd8b00bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:11.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b7769700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd8b00bde20 0x7fd8b0151410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:11.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b7769700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b0066160 con 0x7fd8b00bde20 2023-10-18T17:59:11.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b7769700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b0065cd0 con 0x7fd8b00666b0 2023-10-18T17:59:11.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b7769700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0154750 con 0x7fd8b0067ff0 2023-10-18T17:59:11.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b4d04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd8b00666b0 0x7fd8b00bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:11.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b5d06700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd8b00bde20 0x7fd8b0151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:11.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b4d04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd8b00666b0 0x7fd8b00bd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:41022/0 (socket says 172.21.15.72:41022) 2023-10-18T17:59:11.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b4d04700 1 -- 172.21.15.72:0/1221711988 learned_addr learned my addr 172.21.15.72:0/1221711988 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:11.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b5505700 1 --2- 172.21.15.72:0/1221711988 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b0067ff0 0x7fd8b00683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:11.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b5d06700 1 -- 172.21.15.72:0/1221711988 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b0067ff0 msgr2=0x7fd8b00683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:11.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b5d06700 1 --2- 172.21.15.72:0/1221711988 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b0067ff0 0x7fd8b00683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b5d06700 1 -- 172.21.15.72:0/1221711988 >> v2:172.21.15.179:3300/0 conn(0x7fd8b00666b0 msgr2=0x7fd8b00bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:11.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b5d06700 1 --2- 172.21.15.72:0/1221711988 >> v2:172.21.15.179:3300/0 conn(0x7fd8b00666b0 0x7fd8b00bd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b5d06700 1 -- 172.21.15.72:0/1221711988 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0065100 con 0x7fd8b00bde20 2023-10-18T17:59:11.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b4d04700 1 --2- 172.21.15.72:0/1221711988 >> v2:172.21.15.179:3300/0 conn(0x7fd8b00666b0 0x7fd8b00bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:59:11.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8b5d06700 1 --2- 172.21.15.72:0/1221711988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b00bde20 0x7fd8b0151410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd8b0066d70 tx=0x7fd8a8009780).ready entity=mon.0 client_cookie=999bcf372215b5a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:11.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8a7fff700 1 -- 172.21.15.72:0/1221711988 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800e070 con 0x7fd8b00bde20 2023-10-18T17:59:11.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8a7fff700 1 -- 172.21.15.72:0/1221711988 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a800bab0 con 0x7fd8b00bde20 2023-10-18T17:59:11.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.001+0000 7fd8a7fff700 1 -- 172.21.15.72:0/1221711988 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800be30 con 0x7fd8b00bde20 2023-10-18T17:59:11.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 -- 172.21.15.72:0/1221711988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b00bde20 msgr2=0x7fd8b0151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:11.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 --2- 172.21.15.72:0/1221711988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b00bde20 0x7fd8b0151410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd8b0066d70 tx=0x7fd8a8009780).stop 2023-10-18T17:59:11.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 -- 172.21.15.72:0/1221711988 shutdown_connections 2023-10-18T17:59:11.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 --2- 172.21.15.72:0/1221711988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b00bde20 0x7fd8b0151410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 --2- 172.21.15.72:0/1221711988 >> v2:172.21.15.179:3300/0 conn(0x7fd8b00666b0 0x7fd8b00bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 --2- 172.21.15.72:0/1221711988 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b0067ff0 0x7fd8b00683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 -- 172.21.15.72:0/1221711988 >> 172.21.15.72:0/1221711988 conn(0x7fd8b00620d0 msgr2=0x7fd8b00624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:11.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 -- 172.21.15.72:0/1221711988 shutdown_connections 2023-10-18T17:59:11.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 -- 172.21.15.72:0/1221711988 wait complete. 2023-10-18T17:59:11.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 Processor -- start 2023-10-18T17:59:11.014 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.005+0000 7fd8b7769700 1 -- start start 2023-10-18T17:59:11.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b7769700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b00666b0 0x7fd8b0158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:11.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b7769700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b0067ff0 0x7fd8b0158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:11.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b7769700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b00bde20 0x7fd8b0167490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:11.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b7769700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b015a210 con 0x7fd8b0067ff0 2023-10-18T17:59:11.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b7769700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0159c70 con 0x7fd8b00666b0 2023-10-18T17:59:11.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b7769700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0159f30 con 0x7fd8b00bde20 2023-10-18T17:59:11.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5505700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b00666b0 0x7fd8b0158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:11.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5505700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b00666b0 0x7fd8b0158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:41038/0 (socket says 172.21.15.72:41038) 2023-10-18T17:59:11.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5d06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b00bde20 0x7fd8b0167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:11.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5505700 1 -- 172.21.15.72:0/3268097891 learned_addr learned my addr 172.21.15.72:0/3268097891 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:11.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b4d04700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b0067ff0 0x7fd8b0158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:11.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5d06700 1 -- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b00666b0 msgr2=0x7fd8b0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:11.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5d06700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b00666b0 0x7fd8b0158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5d06700 1 -- 172.21.15.72:0/3268097891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b0067ff0 msgr2=0x7fd8b0158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:11.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5d06700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b0067ff0 0x7fd8b0158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5d06700 1 -- 172.21.15.72:0/3268097891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0009370 con 0x7fd8b00bde20 2023-10-18T17:59:11.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b4d04700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b0067ff0 0x7fd8b0158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:59:11.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b5d06700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b00bde20 0x7fd8b0167490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd8a8014000 tx=0x7fd8a8009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:11.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8a57fa700 1 -- 172.21.15.72:0/3268097891 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8012e30 con 0x7fd8b00bde20 2023-10-18T17:59:11.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8a57fa700 1 -- 172.21.15.72:0/3268097891 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a801d630 con 0x7fd8b00bde20 2023-10-18T17:59:11.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a8009a20 con 0x7fd8b00bde20 2023-10-18T17:59:11.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8a57fa700 1 -- 172.21.15.72:0/3268097891 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a801d9f0 con 0x7fd8b00bde20 2023-10-18T17:59:11.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.009+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b0157c10 con 0x7fd8b00bde20 2023-10-18T17:59:11.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.013+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b006c190 con 0x7fd8b00bde20 2023-10-18T17:59:11.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.013+0000 7fd8a57fa700 1 -- 172.21.15.72:0/3268097891 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd8a8027460 con 0x7fd8b00bde20 2023-10-18T17:59:11.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.017+0000 7fd8a57fa700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd89405a510 0x7fd89405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:11.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.017+0000 7fd8a57fa700 1 -- 172.21.15.72:0/3268097891 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fd8a8077510 con 0x7fd8b00bde20 2023-10-18T17:59:11.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.017+0000 7fd8b5505700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd89405a510 0x7fd89405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:11.022 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.017+0000 7fd8b5505700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd89405a510 0x7fd89405c9d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fd8ac009d30 tx=0x7fd8ac009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:11.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.025+0000 7fd8a57fa700 1 -- 172.21.15.72:0/3268097891 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fd8a804abe0 con 0x7fd8b00bde20 2023-10-18T17:59:11.227 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.221+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd8b00653a0 con 0x7fd8b00bde20 2023-10-18T17:59:11.228 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.221+0000 7fd8a57fa700 1 -- 172.21.15.72:0/3268097891 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fd8a804e200 con 0x7fd8b00bde20 2023-10-18T17:59:11.228 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:11.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd89405a510 msgr2=0x7fd89405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:11.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd89405a510 0x7fd89405c9d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fd8ac009d30 tx=0x7fd8ac009060).stop 2023-10-18T17:59:11.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b00bde20 msgr2=0x7fd8b0167490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:11.233 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b00bde20 0x7fd8b0167490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd8a8014000 tx=0x7fd8a8009780).stop 2023-10-18T17:59:11.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 shutdown_connections 2023-10-18T17:59:11.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd89405a510 0x7fd89405c9d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd8b00bde20 0x7fd8b0167490 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd8b0067ff0 0x7fd8b0158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.234 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 --2- 172.21.15.72:0/3268097891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd8b00666b0 0x7fd8b0158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:11.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 >> 172.21.15.72:0/3268097891 conn(0x7fd8b00620d0 msgr2=0x7fd8b014e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:11.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 shutdown_connections 2023-10-18T17:59:11.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:11.229+0000 7fd8b7769700 1 -- 172.21.15.72:0/3268097891 wait complete. 2023-10-18T17:59:11.256 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:12.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~41s 2023-10-18T17:59:12.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2023-10-18T17:59:14.257 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:14.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.417+0000 7f1a7749b700 1 Processor -- start 2023-10-18T17:59:14.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.421+0000 7f1a7749b700 1 -- start start 2023-10-18T17:59:14.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.421+0000 7f1a7749b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 0x7f1a700bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:14.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.421+0000 7f1a7749b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a70065cd0 con 0x7f1a70067ff0 2023-10-18T17:59:14.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.421+0000 7f1a7749b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a70154750 con 0x7f1a700bde20 2023-10-18T17:59:14.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.421+0000 7f1a7749b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a70066160 con 0x7f1a700666b0 2023-10-18T17:59:14.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.421+0000 7f1a75237700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f1a70067ff0 0x7f1a700683d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:34044/0 (socket says 172.21.15.72:34044) 2023-10-18T17:59:14.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.421+0000 7f1a75237700 1 -- 172.21.15.72:0/3089274252 learned_addr learned my addr 172.21.15.72:0/3089274252 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:14.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.421+0000 7f1a74a36700 1 --2- 172.21.15.72:0/3089274252 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 0x7f1a700bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:14.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a67fff700 1 -- 172.21.15.72:0/3089274252 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258296645 0 0) 0x7f1a70065cd0 con 0x7f1a70067ff0 2023-10-18T17:59:14.429 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a67fff700 1 -- 172.21.15.72:0/3089274252 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a5c0033f0 con 0x7f1a70067ff0 2023-10-18T17:59:14.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a67fff700 1 -- 172.21.15.72:0/3089274252 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919158057 0 0) 0x7f1a70154750 con 0x7f1a700bde20 2023-10-18T17:59:14.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a67fff700 1 -- 172.21.15.72:0/3089274252 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a70065cd0 con 0x7f1a700bde20 2023-10-18T17:59:14.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a74a36700 1 -- 172.21.15.72:0/3089274252 >> v1:172.21.15.179:6789/0 conn(0x7f1a700bde20 legacy=0x7f1a70151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:14.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a74a36700 1 -- 172.21.15.72:0/3089274252 >> v1:172.21.15.72:6789/0 conn(0x7f1a70067ff0 legacy=0x7f1a700683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:14.430 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a74a36700 1 -- 172.21.15.72:0/3089274252 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70065100 con 0x7f1a700666b0 2023-10-18T17:59:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a74a36700 1 --2- 172.21.15.72:0/3089274252 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 0x7f1a700bd930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1a60009790 tx=0x7f1a60009060).ready entity=mon.2 client_cookie=65d1e17f5510dc65 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a67fff700 1 -- 172.21.15.72:0/3089274252 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000f510 con 0x7f1a700666b0 2023-10-18T17:59:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a67fff700 1 -- 172.21.15.72:0/3089274252 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6000f670 con 0x7f1a700666b0 2023-10-18T17:59:14.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a67fff700 1 -- 172.21.15.72:0/3089274252 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000f9f0 con 0x7f1a700666b0 2023-10-18T17:59:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a7749b700 1 -- 172.21.15.72:0/3089274252 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 msgr2=0x7f1a700bd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a7749b700 1 --2- 172.21.15.72:0/3089274252 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 0x7f1a700bd930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f1a60009790 tx=0x7f1a60009060).stop 2023-10-18T17:59:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a7749b700 1 -- 172.21.15.72:0/3089274252 shutdown_connections 2023-10-18T17:59:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a7749b700 1 --2- 172.21.15.72:0/3089274252 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 0x7f1a700bd930 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:14.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a7749b700 1 -- 172.21.15.72:0/3089274252 >> 172.21.15.72:0/3089274252 conn(0x7f1a700620d0 msgr2=0x7f1a700624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:14.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a7749b700 1 -- 172.21.15.72:0/3089274252 shutdown_connections 2023-10-18T17:59:14.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.425+0000 7f1a7749b700 1 -- 172.21.15.72:0/3089274252 wait complete. 2023-10-18T17:59:14.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.429+0000 7f1a7749b700 1 Processor -- start 2023-10-18T17:59:14.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.429+0000 7f1a7749b700 1 -- start start 2023-10-18T17:59:14.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a7749b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 0x7f1a700b8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:14.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a7749b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a70067ff0 0x7f1a700bb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:14.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a7749b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a700bde20 0x7f1a701645c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:14.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a7749b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700b5690 con 0x7f1a70067ff0 2023-10-18T17:59:14.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a7749b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700bc100 con 0x7f1a700bde20 2023-10-18T17:59:14.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a7749b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a70168290 con 0x7f1a700666b0 2023-10-18T17:59:14.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a74a36700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a70067ff0 0x7f1a700bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:14.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a74a36700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a70067ff0 0x7f1a700bb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:50900/0 (socket says 172.21.15.72:50900) 2023-10-18T17:59:14.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a74a36700 1 -- 172.21.15.72:0/3095801939 learned_addr learned my addr 172.21.15.72:0/3095801939 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:14.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a75a38700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a700bde20 0x7f1a701645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:14.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a75237700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 0x7f1a700b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:14.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a74a36700 1 -- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 msgr2=0x7f1a700b8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:14.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a74a36700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 0x7f1a700b8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:14.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a74a36700 1 -- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a700bde20 msgr2=0x7f1a701645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:14.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a74a36700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a700bde20 0x7f1a701645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:14.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a74a36700 1 -- 172.21.15.72:0/3095801939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600092a0 con 0x7f1a70067ff0 2023-10-18T17:59:14.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a74a36700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a70067ff0 0x7f1a700bb350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f1a60009d80 tx=0x7f1a60017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:14.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a657fa700 1 -- 172.21.15.72:0/3095801939 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000fe80 con 0x7f1a70067ff0 2023-10-18T17:59:14.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a657fa700 1 -- 172.21.15.72:0/3095801939 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6000f7e0 con 0x7f1a70067ff0 2023-10-18T17:59:14.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a700bbb40 con 0x7f1a70067ff0 2023-10-18T17:59:14.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a657fa700 1 -- 172.21.15.72:0/3095801939 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6001b670 con 0x7f1a70067ff0 2023-10-18T17:59:14.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.433+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a70167c40 con 0x7f1a70067ff0 2023-10-18T17:59:14.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.437+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7006c190 con 0x7f1a70067ff0 2023-10-18T17:59:14.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.437+0000 7f1a657fa700 1 -- 172.21.15.72:0/3095801939 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1a6001b9e0 con 0x7f1a70067ff0 2023-10-18T17:59:14.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.437+0000 7f1a657fa700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1a5405a510 0x7f1a5405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:14.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.437+0000 7f1a657fa700 1 -- 172.21.15.72:0/3095801939 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f1a60075e60 con 0x7f1a70067ff0 2023-10-18T17:59:14.444 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.437+0000 7f1a75237700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1a5405a510 0x7f1a5405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:14.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.437+0000 7f1a75237700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1a5405a510 0x7f1a5405c9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f1a6c005750 tx=0x7f1a6c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:14.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.445+0000 7f1a657fa700 1 -- 172.21.15.72:0/3095801939 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f1a60049510 con 0x7f1a70067ff0 2023-10-18T17:59:14.644 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1a701683d0 con 0x7f1a70067ff0 2023-10-18T17:59:14.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a657fa700 1 -- 172.21.15.72:0/3095801939 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f1a6004cb30 con 0x7f1a70067ff0 2023-10-18T17:59:14.646 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:14.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1a5405a510 msgr2=0x7f1a5405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:14.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1a5405a510 0x7f1a5405c9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f1a6c005750 tx=0x7f1a6c009040).stop 2023-10-18T17:59:14.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a70067ff0 msgr2=0x7f1a700bb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:14.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a70067ff0 0x7f1a700bb350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f1a60009d80 tx=0x7f1a60017000).stop 2023-10-18T17:59:14.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 shutdown_connections 2023-10-18T17:59:14.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1a5405a510 0x7f1a5405c9d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:14.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a700bde20 0x7f1a701645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:14.649 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a70067ff0 0x7f1a700bb350 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:14.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 --2- 172.21.15.72:0/3095801939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1a700666b0 0x7f1a700b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:14.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.641+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 >> 172.21.15.72:0/3095801939 conn(0x7f1a700620d0 msgr2=0x7f1a70153c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:14.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.645+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 shutdown_connections 2023-10-18T17:59:14.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:14.645+0000 7f1a7749b700 1 -- 172.21.15.72:0/3095801939 wait complete. 2023-10-18T17:59:14.669 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:17.671 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:17.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.825+0000 7fa3dcc39700 1 Processor -- start 2023-10-18T17:59:17.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.825+0000 7fa3dcc39700 1 -- start start 2023-10-18T17:59:17.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3dcc39700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 0x7fa3d80630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:17.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3dcc39700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa3d8068150 0x7fa3d80635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:17.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3dcc39700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d8068910 con 0x7fa3d814d170 2023-10-18T17:59:17.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3dcc39700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d8150c10 con 0x7fa3d8068150 2023-10-18T17:59:17.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3dcc39700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d8150780 con 0x7fa3d8067d30 2023-10-18T17:59:17.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d6ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 0x7fa3d80630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:17.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d7fff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa3d8068150 0x7fa3d80635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:17.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d77fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fa3d814d170 0x7fa3d806a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:50270/0 (socket says 172.21.15.72:50270) 2023-10-18T17:59:17.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d77fe700 1 -- 172.21.15.72:0/1773155354 learned_addr learned my addr 172.21.15.72:0/1773155354 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:17.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d67fc700 1 -- 172.21.15.72:0/1773155354 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209908623 0 0) 0x7fa3d8068910 con 0x7fa3d814d170 2023-10-18T17:59:17.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d67fc700 1 -- 172.21.15.72:0/1773155354 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c00033f0 con 0x7fa3d814d170 2023-10-18T17:59:17.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d6ffd700 1 -- 172.21.15.72:0/1773155354 >> v2:172.21.15.72:3300/0 conn(0x7fa3d8068150 msgr2=0x7fa3d80635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:17.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d6ffd700 1 --2- 172.21.15.72:0/1773155354 >> v2:172.21.15.72:3300/0 conn(0x7fa3d8068150 0x7fa3d80635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:17.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d6ffd700 1 -- 172.21.15.72:0/1773155354 >> v1:172.21.15.179:6789/0 conn(0x7fa3d814d170 legacy=0x7fa3d806a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:17.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d6ffd700 1 -- 172.21.15.72:0/1773155354 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d8068630 con 0x7fa3d8067d30 2023-10-18T17:59:17.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d7fff700 1 --2- 172.21.15.72:0/1773155354 >> v2:172.21.15.72:3300/0 conn(0x7fa3d8068150 0x7fa3d80635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:59:17.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d6ffd700 1 --2- 172.21.15.72:0/1773155354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 0x7fa3d80630e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa3cc009760 tx=0x7fa3cc009060).ready entity=mon.2 client_cookie=3f74e4232888e127 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:17.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d67fc700 1 -- 172.21.15.72:0/1773155354 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00f510 con 0x7fa3d8067d30 2023-10-18T17:59:17.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d67fc700 1 -- 172.21.15.72:0/1773155354 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc00f670 con 0x7fa3d8067d30 2023-10-18T17:59:17.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.829+0000 7fa3d67fc700 1 -- 172.21.15.72:0/1773155354 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00f9f0 con 0x7fa3d8067d30 2023-10-18T17:59:17.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1773155354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 msgr2=0x7fa3d80630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:17.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 --2- 172.21.15.72:0/1773155354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 0x7fa3d80630e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa3cc009760 tx=0x7fa3cc009060).stop 2023-10-18T17:59:17.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1773155354 shutdown_connections 2023-10-18T17:59:17.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 --2- 172.21.15.72:0/1773155354 >> v2:172.21.15.72:3300/0 conn(0x7fa3d8068150 0x7fa3d80635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:17.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 --2- 172.21.15.72:0/1773155354 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 0x7fa3d80630e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:17.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1773155354 >> 172.21.15.72:0/1773155354 conn(0x7fa3d8062ba0 msgr2=0x7fa3d80c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:17.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1773155354 shutdown_connections 2023-10-18T17:59:17.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1773155354 wait complete. 2023-10-18T17:59:17.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 Processor -- start 2023-10-18T17:59:17.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.833+0000 7fa3dcc39700 1 -- start start 2023-10-18T17:59:17.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3dcc39700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 0x7fa3d80c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:17.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3dcc39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d8068150 0x7fa3d814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:17.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3dcc39700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3d814d170 0x7fa3d80bc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:17.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3dcc39700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d8154420 con 0x7fa3d814d170 2023-10-18T17:59:17.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3dcc39700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81517f0 con 0x7fa3d8068150 2023-10-18T17:59:17.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3dcc39700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d80be800 con 0x7fa3d8067d30 2023-10-18T17:59:17.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d7fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3d814d170 0x7fa3d80bc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:17.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d77fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 0x7fa3d80c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:17.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d7fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3d814d170 0x7fa3d80bc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:50914/0 (socket says 172.21.15.72:50914) 2023-10-18T17:59:17.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d7fff700 1 -- 172.21.15.72:0/1681461637 learned_addr learned my addr 172.21.15.72:0/1681461637 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:17.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d6ffd700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d8068150 0x7fa3d814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:17.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d7fff700 1 -- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 msgr2=0x7fa3d80c06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:17.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d7fff700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 0x7fa3d80c06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:17.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d7fff700 1 -- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d8068150 msgr2=0x7fa3d814beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:17.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d7fff700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d8068150 0x7fa3d814beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:17.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d7fff700 1 -- 172.21.15.72:0/1681461637 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0009a20 con 0x7fa3d814d170 2023-10-18T17:59:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3d7fff700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3d814d170 0x7fa3d80bc640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fa3d0009f60 tx=0x7fa3d000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3bf7fe700 1 -- 172.21.15.72:0/1681461637 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0009d00 con 0x7fa3d814d170 2023-10-18T17:59:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3bf7fe700 1 -- 172.21.15.72:0/1681461637 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d000f040 con 0x7fa3d814d170 2023-10-18T17:59:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc0092a0 con 0x7fa3d814d170 2023-10-18T17:59:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3bf7fe700 1 -- 172.21.15.72:0/1681461637 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000cda0 con 0x7fa3d814d170 2023-10-18T17:59:17.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.837+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d8151230 con 0x7fa3d814d170 2023-10-18T17:59:17.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.841+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d8046f00 con 0x7fa3d814d170 2023-10-18T17:59:17.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.841+0000 7fa3bf7fe700 1 -- 172.21.15.72:0/1681461637 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa3d0013610 con 0x7fa3d814d170 2023-10-18T17:59:17.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.841+0000 7fa3bf7fe700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa3c405a510 0x7fa3c405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:17.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.841+0000 7fa3bf7fe700 1 -- 172.21.15.72:0/1681461637 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fa3d00750f0 con 0x7fa3d814d170 2023-10-18T17:59:17.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.841+0000 7fa3d77fe700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa3c405a510 0x7fa3c405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:17.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.845+0000 7fa3d77fe700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa3c405a510 0x7fa3c405c9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fa3c8005750 tx=0x7fa3c8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:17.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:17.849+0000 7fa3bf7fe700 1 -- 172.21.15.72:0/1681461637 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fa3d00487a0 con 0x7fa3d814d170 2023-10-18T17:59:18.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.045+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa3d814fe50 con 0x7fa3d814d170 2023-10-18T17:59:18.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.045+0000 7fa3bf7fe700 1 -- 172.21.15.72:0/1681461637 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fa3d004bdc0 con 0x7fa3d814d170 2023-10-18T17:59:18.052 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:18.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa3c405a510 msgr2=0x7fa3c405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:18.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa3c405a510 0x7fa3c405c9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fa3c8005750 tx=0x7fa3c8009040).stop 2023-10-18T17:59:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3d814d170 msgr2=0x7fa3d80bc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3d814d170 0x7fa3d80bc640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fa3d0009f60 tx=0x7fa3d000b040).stop 2023-10-18T17:59:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 shutdown_connections 2023-10-18T17:59:18.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa3c405a510 0x7fa3c405c9d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:18.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3d814d170 0x7fa3d80bc640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:18.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3d8068150 0x7fa3d814beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:18.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 --2- 172.21.15.72:0/1681461637 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa3d8067d30 0x7fa3d80c06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:18.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.049+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 >> 172.21.15.72:0/1681461637 conn(0x7fa3d8062ba0 msgr2=0x7fa3d8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:18.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.053+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 shutdown_connections 2023-10-18T17:59:18.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:18.053+0000 7fa3dcc39700 1 -- 172.21.15.72:0/1681461637 wait complete. 2023-10-18T17:59:18.077 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:19.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~48s 2023-10-18T17:59:19.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~7s 2023-10-18T17:59:21.078 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:21.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.229+0000 7fbd50b07700 1 Processor -- start 2023-10-18T17:59:21.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.229+0000 7fbd50b07700 1 -- start start 2023-10-18T17:59:21.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd50b07700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0c2bf0 0x7fbd4c065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:21.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd50b07700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fbd4c0bd1c0 0x7fbd4c14ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:21.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd50b07700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4c0be2e0 con 0x7fbd4c0658a0 2023-10-18T17:59:21.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd50b07700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd4c0bda80 con 0x7fbd4c0bd1c0 2023-10-18T17:59:21.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd50b07700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c06a7c0 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4b7fe700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fbd4c0658a0 0x7fbd4c0630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:57550/0 (socket says 172.21.15.72:57550) 2023-10-18T17:59:21.238 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4b7fe700 1 -- 172.21.15.72:0/3338735398 learned_addr learned my addr 172.21.15.72:0/3338735398 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:21.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4bfff700 1 --2- 172.21.15.72:0/3338735398 >> v2:172.21.15.72:3300/0 conn(0x7fbd4c0bd1c0 0x7fbd4c14ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:21.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4affd700 1 --2- 172.21.15.72:0/3338735398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0c2bf0 0x7fbd4c065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:21.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4a7fc700 1 -- 172.21.15.72:0/3338735398 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791742549 0 0) 0x7fbd4c0be2e0 con 0x7fbd4c0658a0 2023-10-18T17:59:21.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4a7fc700 1 -- 172.21.15.72:0/3338735398 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd340033f0 con 0x7fbd4c0658a0 2023-10-18T17:59:21.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4bfff700 1 -- 172.21.15.72:0/3338735398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0c2bf0 msgr2=0x7fbd4c065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:21.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4bfff700 1 --2- 172.21.15.72:0/3338735398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0c2bf0 0x7fbd4c065c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4bfff700 1 -- 172.21.15.72:0/3338735398 >> v1:172.21.15.72:6789/0 conn(0x7fbd4c0658a0 legacy=0x7fbd4c0630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4bfff700 1 -- 172.21.15.72:0/3338735398 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c0be770 con 0x7fbd4c0bd1c0 2023-10-18T17:59:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4bfff700 1 --2- 172.21.15.72:0/3338735398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0bd1c0 0x7fbd4c14ddb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fbd44009e60 tx=0x7fbd44009780).ready entity=mon.0 client_cookie=eba2bc17e34e6ac7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4a7fc700 1 -- 172.21.15.72:0/3338735398 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400e070 con 0x7fbd4c0bd1c0 2023-10-18T17:59:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4a7fc700 1 -- 172.21.15.72:0/3338735398 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4400bab0 con 0x7fbd4c0bd1c0 2023-10-18T17:59:21.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.233+0000 7fbd4a7fc700 1 -- 172.21.15.72:0/3338735398 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400be30 con 0x7fbd4c0bd1c0 2023-10-18T17:59:21.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.237+0000 7fbd50b07700 1 -- 172.21.15.72:0/3338735398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0bd1c0 msgr2=0x7fbd4c14ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:21.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.237+0000 7fbd50b07700 1 --2- 172.21.15.72:0/3338735398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0bd1c0 0x7fbd4c14ddb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fbd44009e60 tx=0x7fbd44009780).stop 2023-10-18T17:59:21.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.237+0000 7fbd50b07700 1 -- 172.21.15.72:0/3338735398 shutdown_connections 2023-10-18T17:59:21.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.237+0000 7fbd50b07700 1 --2- 172.21.15.72:0/3338735398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0bd1c0 0x7fbd4c14ddb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:21.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.237+0000 7fbd50b07700 1 --2- 172.21.15.72:0/3338735398 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0c2bf0 0x7fbd4c065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:21.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.237+0000 7fbd50b07700 1 -- 172.21.15.72:0/3338735398 >> 172.21.15.72:0/3338735398 conn(0x7fbd4c062ba0 msgr2=0x7fbd4c0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:21.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.237+0000 7fbd50b07700 1 -- 172.21.15.72:0/3338735398 shutdown_connections 2023-10-18T17:59:21.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.237+0000 7fbd50b07700 1 -- 172.21.15.72:0/3338735398 wait complete. 2023-10-18T17:59:21.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd50b07700 1 Processor -- start 2023-10-18T17:59:21.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd50b07700 1 -- start start 2023-10-18T17:59:21.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd50b07700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0658a0 0x7fbd4c1640b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:21.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd50b07700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd4c0bd1c0 0x7fbd4c14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:21.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd50b07700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0c2bf0 0x7fbd4c14c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:21.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd50b07700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c1645a0 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd50b07700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0bbd60 con 0x7fbd4c0bd1c0 2023-10-18T17:59:21.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd50b07700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0bc020 con 0x7fbd4c0658a0 2023-10-18T17:59:21.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0c2bf0 0x7fbd4c14c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:21.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4b7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0658a0 0x7fbd4c1640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:21.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0c2bf0 0x7fbd4c14c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44976/0 (socket says 172.21.15.72:44976) 2023-10-18T17:59:21.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4bfff700 1 -- 172.21.15.72:0/1232009810 learned_addr learned my addr 172.21.15.72:0/1232009810 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:21.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4affd700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd4c0bd1c0 0x7fbd4c14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:21.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4bfff700 1 -- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0658a0 msgr2=0x7fbd4c1640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:21.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4bfff700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0658a0 0x7fbd4c1640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:21.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4bfff700 1 -- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd4c0bd1c0 msgr2=0x7fbd4c14bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:21.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4bfff700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd4c0bd1c0 0x7fbd4c14bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:21.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.241+0000 7fbd4bfff700 1 -- 172.21.15.72:0/1232009810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44009a20 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.245+0000 7fbd4bfff700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0c2bf0 0x7fbd4c14c4b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fbd4400b610 tx=0x7fbd44009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:21.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.245+0000 7fbd337fe700 1 -- 172.21.15.72:0/1232009810 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44013e70 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.245+0000 7fbd337fe700 1 -- 172.21.15.72:0/1232009810 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4401d630 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.245+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c0a48a0 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.245+0000 7fbd337fe700 1 -- 172.21.15.72:0/1232009810 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4401d9f0 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.245+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c0a4e60 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.245+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4c0bc860 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.245+0000 7fbd337fe700 1 -- 172.21.15.72:0/1232009810 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbd44003040 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.249+0000 7fbd337fe700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbd3805a510 0x7fbd3805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:21.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.249+0000 7fbd337fe700 1 -- 172.21.15.72:0/1232009810 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fbd440766a0 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.249+0000 7fbd4b7fe700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbd3805a510 0x7fbd3805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:21.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.249+0000 7fbd4b7fe700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbd3805a510 0x7fbd3805c9d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fbd3c007fd0 tx=0x7fbd3c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:21.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.257+0000 7fbd337fe700 1 -- 172.21.15.72:0/1232009810 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fbd4c0bc860 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.445+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbd4c0bc860 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd337fe700 1 -- 172.21.15.72:0/1232009810 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fbd4c0bc860 con 0x7fbd4c0c2bf0 2023-10-18T17:59:21.453 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:21.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbd3805a510 msgr2=0x7fbd3805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:21.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbd3805a510 0x7fbd3805c9d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fbd3c007fd0 tx=0x7fbd3c009040).stop 2023-10-18T17:59:21.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0c2bf0 msgr2=0x7fbd4c14c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:21.456 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0c2bf0 0x7fbd4c14c4b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fbd4400b610 tx=0x7fbd44009780).stop 2023-10-18T17:59:21.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 shutdown_connections 2023-10-18T17:59:21.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbd3805a510 0x7fbd3805c9d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:21.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd4c0c2bf0 0x7fbd4c14c4b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:21.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd4c0bd1c0 0x7fbd4c14bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:21.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 --2- 172.21.15.72:0/1232009810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbd4c0658a0 0x7fbd4c1640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:21.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 >> 172.21.15.72:0/1232009810 conn(0x7fbd4c062ba0 msgr2=0x7fbd4c0baa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:21.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 shutdown_connections 2023-10-18T17:59:21.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:21.449+0000 7fbd50b07700 1 -- 172.21.15.72:0/1232009810 wait complete. 2023-10-18T17:59:21.474 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:24.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:24.553 INFO:tasks.ceph.mds.c.smithi072.stderr:terminate called after throwing an instance of 'boost::wrapexcept' 2023-10-18T17:59:24.554 INFO:tasks.ceph.mds.c.smithi072.stderr: what(): boost::bad_get: failed value get using boost::get 2023-10-18T17:59:24.554 INFO:tasks.ceph.mds.c.smithi072.stderr:*** Caught signal (Aborted) ** 2023-10-18T17:59:24.554 INFO:tasks.ceph.mds.c.smithi072.stderr: in thread 7f581dee1700 thread_name:ms_dispatch 2023-10-18T17:59:24.555 INFO:tasks.ceph.mds.c.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:24.555 INFO:tasks.ceph.mds.c.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f582be72420] 2023-10-18T17:59:24.555 INFO:tasks.ceph.mds.c.smithi072.stderr: 2: gsignal() 2023-10-18T17:59:24.555 INFO:tasks.ceph.mds.c.smithi072.stderr: 3: abort() 2023-10-18T17:59:24.556 INFO:tasks.ceph.mds.c.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f5822ebc911] 2023-10-18T17:59:24.556 INFO:tasks.ceph.mds.c.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f5822ec838c] 2023-10-18T17:59:24.556 INFO:tasks.ceph.mds.c.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f5822ec83f7] 2023-10-18T17:59:24.556 INFO:tasks.ceph.mds.c.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f5822ec86a9] 2023-10-18T17:59:24.556 INFO:tasks.ceph.mds.c.smithi072.stderr: 8: ceph-mds(+0x185107) [0x5617cd65b107] 2023-10-18T17:59:24.557 INFO:tasks.ceph.mds.c.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x5617cd65b72d] 2023-10-18T17:59:24.557 INFO:tasks.ceph.mds.c.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5617cd6cd09f] 2023-10-18T17:59:24.557 INFO:tasks.ceph.mds.c.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5617cd6cea95] 2023-10-18T17:59:24.557 INFO:tasks.ceph.mds.c.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5617cd699b10] 2023-10-18T17:59:24.557 INFO:tasks.ceph.mds.c.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5617cd69dabf] 2023-10-18T17:59:24.558 INFO:tasks.ceph.mds.c.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5617cd69e456] 2023-10-18T17:59:24.558 INFO:tasks.ceph.mds.c.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f58234bc710] 2023-10-18T17:59:24.558 INFO:tasks.ceph.mds.c.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f58234b9faf] 2023-10-18T17:59:24.558 INFO:tasks.ceph.mds.c.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5823588b21] 2023-10-18T17:59:24.558 INFO:tasks.ceph.mds.c.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f582be66609] 2023-10-18T17:59:24.559 INFO:tasks.ceph.mds.c.smithi072.stderr: 19: clone() 2023-10-18T17:59:24.559 INFO:tasks.ceph.mds.c.smithi072.stderr:2023-10-18T17:59:24.549+0000 7f581dee1700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:24.559 INFO:tasks.ceph.mds.c.smithi072.stderr: in thread 7f581dee1700 thread_name:ms_dispatch 2023-10-18T17:59:24.559 INFO:tasks.ceph.mds.c.smithi072.stderr: 2023-10-18T17:59:24.559 INFO:tasks.ceph.mds.c.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:24.560 INFO:tasks.ceph.mds.c.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f582be72420] 2023-10-18T17:59:24.560 INFO:tasks.ceph.mds.c.smithi072.stderr: 2: gsignal() 2023-10-18T17:59:24.560 INFO:tasks.ceph.mds.c.smithi072.stderr: 3: abort() 2023-10-18T17:59:24.560 INFO:tasks.ceph.mds.c.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f5822ebc911] 2023-10-18T17:59:24.560 INFO:tasks.ceph.mds.c.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f5822ec838c] 2023-10-18T17:59:24.561 INFO:tasks.ceph.mds.c.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f5822ec83f7] 2023-10-18T17:59:24.561 INFO:tasks.ceph.mds.c.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f5822ec86a9] 2023-10-18T17:59:24.561 INFO:tasks.ceph.mds.c.smithi072.stderr: 8: ceph-mds(+0x185107) [0x5617cd65b107] 2023-10-18T17:59:24.561 INFO:tasks.ceph.mds.c.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x5617cd65b72d] 2023-10-18T17:59:24.561 INFO:tasks.ceph.mds.c.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5617cd6cd09f] 2023-10-18T17:59:24.562 INFO:tasks.ceph.mds.c.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5617cd6cea95] 2023-10-18T17:59:24.562 INFO:tasks.ceph.mds.c.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5617cd699b10] 2023-10-18T17:59:24.562 INFO:tasks.ceph.mds.c.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5617cd69dabf] 2023-10-18T17:59:24.562 INFO:tasks.ceph.mds.c.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5617cd69e456] 2023-10-18T17:59:24.563 INFO:tasks.ceph.mds.c.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f58234bc710] 2023-10-18T17:59:24.563 INFO:tasks.ceph.mds.c.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f58234b9faf] 2023-10-18T17:59:24.563 INFO:tasks.ceph.mds.c.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5823588b21] 2023-10-18T17:59:24.563 INFO:tasks.ceph.mds.c.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f582be66609] 2023-10-18T17:59:24.563 INFO:tasks.ceph.mds.c.smithi072.stderr: 19: clone() 2023-10-18T17:59:24.564 INFO:tasks.ceph.mds.c.smithi072.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:24.564 INFO:tasks.ceph.mds.c.smithi072.stderr: 2023-10-18T17:59:24.564 INFO:tasks.ceph.mds.c.smithi072.stderr: 0> 2023-10-18T17:59:24.549+0000 7f581dee1700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:24.565 INFO:tasks.ceph.mds.c.smithi072.stderr: in thread 7f581dee1700 thread_name:ms_dispatch 2023-10-18T17:59:24.565 INFO:tasks.ceph.mds.c.smithi072.stderr: 2023-10-18T17:59:24.565 INFO:tasks.ceph.mds.c.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:24.565 INFO:tasks.ceph.mds.c.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f582be72420] 2023-10-18T17:59:24.565 INFO:tasks.ceph.mds.c.smithi072.stderr: 2: gsignal() 2023-10-18T17:59:24.566 INFO:tasks.ceph.mds.c.smithi072.stderr: 3: abort() 2023-10-18T17:59:24.566 INFO:tasks.ceph.mds.c.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f5822ebc911] 2023-10-18T17:59:24.566 INFO:tasks.ceph.mds.c.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f5822ec838c] 2023-10-18T17:59:24.566 INFO:tasks.ceph.mds.c.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f5822ec83f7] 2023-10-18T17:59:24.566 INFO:tasks.ceph.mds.c.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f5822ec86a9] 2023-10-18T17:59:24.567 INFO:tasks.ceph.mds.c.smithi072.stderr: 8: ceph-mds(+0x185107) [0x5617cd65b107] 2023-10-18T17:59:24.567 INFO:tasks.ceph.mds.c.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x5617cd65b72d] 2023-10-18T17:59:24.567 INFO:tasks.ceph.mds.c.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5617cd6cd09f] 2023-10-18T17:59:24.567 INFO:tasks.ceph.mds.c.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5617cd6cea95] 2023-10-18T17:59:24.567 INFO:tasks.ceph.mds.c.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5617cd699b10] 2023-10-18T17:59:24.568 INFO:tasks.ceph.mds.c.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5617cd69dabf] 2023-10-18T17:59:24.568 INFO:tasks.ceph.mds.c.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5617cd69e456] 2023-10-18T17:59:24.568 INFO:tasks.ceph.mds.c.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f58234bc710] 2023-10-18T17:59:24.568 INFO:tasks.ceph.mds.c.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f58234b9faf] 2023-10-18T17:59:24.568 INFO:tasks.ceph.mds.c.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5823588b21] 2023-10-18T17:59:24.569 INFO:tasks.ceph.mds.c.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f582be66609] 2023-10-18T17:59:24.569 INFO:tasks.ceph.mds.c.smithi072.stderr: 19: clone() 2023-10-18T17:59:24.569 INFO:tasks.ceph.mds.c.smithi072.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:24.569 INFO:tasks.ceph.mds.c.smithi072.stderr: 2023-10-18T17:59:24.569 INFO:tasks.ceph.mds.c.smithi072.stderr: 0> 2023-10-18T17:59:24.549+0000 7f581dee1700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:24.570 INFO:tasks.ceph.mds.c.smithi072.stderr: in thread 7f581dee1700 thread_name:ms_dispatch 2023-10-18T17:59:24.570 INFO:tasks.ceph.mds.c.smithi072.stderr: 2023-10-18T17:59:24.570 INFO:tasks.ceph.mds.c.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:24.570 INFO:tasks.ceph.mds.c.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f582be72420] 2023-10-18T17:59:24.570 INFO:tasks.ceph.mds.c.smithi072.stderr: 2: gsignal() 2023-10-18T17:59:24.571 INFO:tasks.ceph.mds.c.smithi072.stderr: 3: abort() 2023-10-18T17:59:24.571 INFO:tasks.ceph.mds.c.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f5822ebc911] 2023-10-18T17:59:24.571 INFO:tasks.ceph.mds.c.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f5822ec838c] 2023-10-18T17:59:24.571 INFO:tasks.ceph.mds.c.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f5822ec83f7] 2023-10-18T17:59:24.571 INFO:tasks.ceph.mds.c.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f5822ec86a9] 2023-10-18T17:59:24.572 INFO:tasks.ceph.mds.c.smithi072.stderr: 8: ceph-mds(+0x185107) [0x5617cd65b107] 2023-10-18T17:59:24.572 INFO:tasks.ceph.mds.c.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x5617cd65b72d] 2023-10-18T17:59:24.572 INFO:tasks.ceph.mds.c.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5617cd6cd09f] 2023-10-18T17:59:24.572 INFO:tasks.ceph.mds.c.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5617cd6cea95] 2023-10-18T17:59:24.572 INFO:tasks.ceph.mds.c.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5617cd699b10] 2023-10-18T17:59:24.573 INFO:tasks.ceph.mds.c.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5617cd69dabf] 2023-10-18T17:59:24.573 INFO:tasks.ceph.mds.c.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5617cd69e456] 2023-10-18T17:59:24.573 INFO:tasks.ceph.mds.c.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f58234bc710] 2023-10-18T17:59:24.573 INFO:tasks.ceph.mds.c.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f58234b9faf] 2023-10-18T17:59:24.574 INFO:tasks.ceph.mds.c.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5823588b21] 2023-10-18T17:59:24.574 INFO:tasks.ceph.mds.c.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f582be66609] 2023-10-18T17:59:24.574 INFO:tasks.ceph.mds.c.smithi072.stderr: 19: clone() 2023-10-18T17:59:24.574 INFO:tasks.ceph.mds.c.smithi072.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:24.575 INFO:tasks.ceph.mds.c.smithi072.stderr: 2023-10-18T17:59:24.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.641+0000 7ff56e3b9700 1 Processor -- start 2023-10-18T17:59:24.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.641+0000 7ff56e3b9700 1 -- start start 2023-10-18T17:59:24.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56e3b9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff568065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:24.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56e3b9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5680be2e0 con 0x7ff5680bd1c0 2023-10-18T17:59:24.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56e3b9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5680be770 con 0x7ff5680c2bf0 2023-10-18T17:59:24.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56e3b9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680bda80 con 0x7ff5680658a0 2023-10-18T17:59:24.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56cbb6700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7ff5680c2bf0 0x7ff5680630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:46214/0 (socket says 172.21.15.72:46214) 2023-10-18T17:59:24.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56cbb6700 1 -- 172.21.15.72:0/4103275488 learned_addr learned my addr 172.21.15.72:0/4103275488 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:24.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56d3b7700 1 --2- 172.21.15.72:0/4103275488 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff568065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:24.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff55ffff700 1 -- 172.21.15.72:0/4103275488 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212221538 0 0) 0x7ff5680be2e0 con 0x7ff5680bd1c0 2023-10-18T17:59:24.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff55ffff700 1 -- 172.21.15.72:0/4103275488 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5540033f0 con 0x7ff5680bd1c0 2023-10-18T17:59:24.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff55ffff700 1 -- 172.21.15.72:0/4103275488 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792674741 0 0) 0x7ff5680be770 con 0x7ff5680c2bf0 2023-10-18T17:59:24.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff55ffff700 1 -- 172.21.15.72:0/4103275488 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5680be2e0 con 0x7ff5680c2bf0 2023-10-18T17:59:24.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56d3b7700 1 -- 172.21.15.72:0/4103275488 >> v1:172.21.15.179:6789/0 conn(0x7ff5680c2bf0 legacy=0x7ff5680630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:24.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56d3b7700 1 -- 172.21.15.72:0/4103275488 >> v1:172.21.15.72:6789/0 conn(0x7ff5680bd1c0 legacy=0x7ff56814ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:24.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.645+0000 7ff56d3b7700 1 -- 172.21.15.72:0/4103275488 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680b9580 con 0x7ff5680658a0 2023-10-18T17:59:24.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff56d3b7700 1 --2- 172.21.15.72:0/4103275488 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff568065c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff564009790 tx=0x7ff564009060).ready entity=mon.2 client_cookie=a3d22a6be6783c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:24.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff55ffff700 1 -- 172.21.15.72:0/4103275488 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56400f510 con 0x7ff5680658a0 2023-10-18T17:59:24.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff55ffff700 1 -- 172.21.15.72:0/4103275488 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56400f670 con 0x7ff5680658a0 2023-10-18T17:59:24.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff55ffff700 1 -- 172.21.15.72:0/4103275488 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56400f9f0 con 0x7ff5680658a0 2023-10-18T17:59:24.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff56e3b9700 1 -- 172.21.15.72:0/4103275488 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 msgr2=0x7ff568065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:24.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff56e3b9700 1 --2- 172.21.15.72:0/4103275488 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff568065c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ff564009790 tx=0x7ff564009060).stop 2023-10-18T17:59:24.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff56e3b9700 1 -- 172.21.15.72:0/4103275488 shutdown_connections 2023-10-18T17:59:24.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff56e3b9700 1 --2- 172.21.15.72:0/4103275488 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff568065c80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:24.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff56e3b9700 1 -- 172.21.15.72:0/4103275488 >> 172.21.15.72:0/4103275488 conn(0x7ff568062ba0 msgr2=0x7ff5680c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:24.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff56e3b9700 1 -- 172.21.15.72:0/4103275488 shutdown_connections 2023-10-18T17:59:24.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.649+0000 7ff56e3b9700 1 -- 172.21.15.72:0/4103275488 wait complete. 2023-10-18T17:59:24.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56e3b9700 1 Processor -- start 2023-10-18T17:59:24.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56e3b9700 1 -- start start 2023-10-18T17:59:24.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56e3b9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff5680c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:24.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56e3b9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5680bd1c0 0x7ff5680b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:24.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56e3b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5680c2bf0 0x7ff568148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:24.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56e3b9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680c1940 con 0x7ff5680c2bf0 2023-10-18T17:59:24.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56e3b9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff568160a40 con 0x7ff5680bd1c0 2023-10-18T17:59:24.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56e3b9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56814b1e0 con 0x7ff5680658a0 2023-10-18T17:59:24.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56d3b7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff5680c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:24.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56dbb8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5680c2bf0 0x7ff568148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:24.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56cbb6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5680bd1c0 0x7ff5680b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:24.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56d3b7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff5680c2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:60544/0 (socket says 172.21.15.72:60544) 2023-10-18T17:59:24.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56d3b7700 1 -- 172.21.15.72:0/162214571 learned_addr learned my addr 172.21.15.72:0/162214571 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:24.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56dbb8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5680c2bf0 0x7ff568148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44992/0 (socket says 172.21.15.72:44992) 2023-10-18T17:59:24.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56dbb8700 1 -- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 msgr2=0x7ff5680c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:24.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56dbb8700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff5680c2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:24.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56dbb8700 1 -- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5680bd1c0 msgr2=0x7ff5680b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:24.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56dbb8700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5680bd1c0 0x7ff5680b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:24.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56dbb8700 1 -- 172.21.15.72:0/162214571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640092a0 con 0x7ff5680c2bf0 2023-10-18T17:59:24.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.653+0000 7ff56dbb8700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5680c2bf0 0x7ff568148f90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ff56000c790 tx=0x7ff560007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:24.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.657+0000 7ff55d7fa700 1 -- 172.21.15.72:0/162214571 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56000e4a0 con 0x7ff5680c2bf0 2023-10-18T17:59:24.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.657+0000 7ff55d7fa700 1 -- 172.21.15.72:0/162214571 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56000e600 con 0x7ff5680c2bf0 2023-10-18T17:59:24.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.657+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5680c2170 con 0x7ff5680c2bf0 2023-10-18T17:59:24.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.657+0000 7ff55d7fa700 1 -- 172.21.15.72:0/162214571 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56000e9c0 con 0x7ff5680c2bf0 2023-10-18T17:59:24.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.657+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff568160780 con 0x7ff5680c2bf0 2023-10-18T17:59:24.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.657+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5680b5fe0 con 0x7ff5680c2bf0 2023-10-18T17:59:24.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.657+0000 7ff55d7fa700 1 -- 172.21.15.72:0/162214571 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff56001b3b0 con 0x7ff5680c2bf0 2023-10-18T17:59:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.657+0000 7ff55d7fa700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff54c05a540 0x7ff54c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.661+0000 7ff55d7fa700 1 -- 172.21.15.72:0/162214571 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6322+0+0 (secure 0 0 0) 0x7ff560074090 con 0x7ff5680c2bf0 2023-10-18T17:59:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.661+0000 7ff56d3b7700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff54c05a540 0x7ff54c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:24.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.661+0000 7ff56d3b7700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff54c05a540 0x7ff54c05ca00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7ff564009fd0 tx=0x7ff564017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:24.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.665+0000 7ff55d7fa700 1 -- 172.21.15.72:0/162214571 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ff5680b5fe0 con 0x7ff5680c2bf0 2023-10-18T17:59:24.814 INFO:tasks.ceph.mds.c.smithi072.stderr:daemon-helper: command crashed with signal 6 2023-10-18T17:59:24.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.853+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff5680b5fe0 con 0x7ff5680c2bf0 2023-10-18T17:59:24.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff55d7fa700 1 -- 172.21.15.72:0/162214571 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7ff5680b5fe0 con 0x7ff5680c2bf0 2023-10-18T17:59:24.861 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:24.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff54c05a540 msgr2=0x7ff54c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:24.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff54c05a540 0x7ff54c05ca00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7ff564009fd0 tx=0x7ff564017000).stop 2023-10-18T17:59:24.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5680c2bf0 msgr2=0x7ff568148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:24.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5680c2bf0 0x7ff568148f90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7ff56000c790 tx=0x7ff560007f10).stop 2023-10-18T17:59:24.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 shutdown_connections 2023-10-18T17:59:24.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff54c05a540 0x7ff54c05ca00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:24.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff5680c2bf0 0x7ff568148f90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:24.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5680bd1c0 0x7ff5680b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:24.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 --2- 172.21.15.72:0/162214571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff5680658a0 0x7ff5680c2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:24.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 >> 172.21.15.72:0/162214571 conn(0x7ff568062ba0 msgr2=0x7ff5680bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:24.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.857+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 shutdown_connections 2023-10-18T17:59:24.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:24.861+0000 7ff56e3b9700 1 -- 172.21.15.72:0/162214571 wait complete. 2023-10-18T17:59:24.883 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:25.746 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~54s 2023-10-18T17:59:25.746 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-10-18T17:59:25.746 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~13s 2023-10-18T17:59:26.474 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:59:26.481 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:59:27.884 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:28.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f988da79700 1 Processor -- start 2023-10-18T17:59:28.045 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f988da79700 1 -- start start 2023-10-18T17:59:28.047 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f988da79700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f98881513e0 0x7f9888067d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:28.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f988da79700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9888154b00 con 0x7f98880630e0 2023-10-18T17:59:28.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f988da79700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988806a750 con 0x7f98881517c0 2023-10-18T17:59:28.048 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f988da79700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9888069380 con 0x7f98881513e0 2023-10-18T17:59:28.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f988ca77700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f98881513e0 0x7f9888067d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:28.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f987ffff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f98881517c0 0x7f988806a040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:46220/0 (socket says 172.21.15.72:46220) 2023-10-18T17:59:28.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f988d278700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f98880630e0 0x7f9888065700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:57580/0 (socket says 172.21.15.72:57580) 2023-10-18T17:59:28.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.041+0000 7f987ffff700 1 -- 172.21.15.72:0/2566421159 learned_addr learned my addr 172.21.15.72:0/2566421159 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:28.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988ca77700 1 -- 172.21.15.72:0/2566421159 >> v1:172.21.15.179:6789/0 conn(0x7f98881517c0 legacy=0x7f988806a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:28.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988ca77700 1 -- 172.21.15.72:0/2566421159 >> v1:172.21.15.72:6789/0 conn(0x7f98880630e0 legacy=0x7f9888065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:28.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988ca77700 1 -- 172.21.15.72:0/2566421159 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9888154f90 con 0x7f98881513e0 2023-10-18T17:59:28.050 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f987f7fe700 1 -- 172.21.15.72:0/2566421159 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017622301 0 0) 0x7f9888154b00 con 0x7f98880630e0 2023-10-18T17:59:28.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988ca77700 1 --2- 172.21.15.72:0/2566421159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881513e0 0x7f9888067d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f9884009760 tx=0x7f9884009060).ready entity=mon.0 client_cookie=425e509672815808 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:28.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f987f7fe700 1 -- 172.21.15.72:0/2566421159 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988400b610 con 0x7f98881513e0 2023-10-18T17:59:28.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f987f7fe700 1 -- 172.21.15.72:0/2566421159 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988400b770 con 0x7f98881513e0 2023-10-18T17:59:28.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f987f7fe700 1 -- 172.21.15.72:0/2566421159 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988400baf0 con 0x7f98881513e0 2023-10-18T17:59:28.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988da79700 1 -- 172.21.15.72:0/2566421159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881513e0 msgr2=0x7f9888067d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:28.056 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988da79700 1 --2- 172.21.15.72:0/2566421159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881513e0 0x7f9888067d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f9884009760 tx=0x7f9884009060).stop 2023-10-18T17:59:28.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988da79700 1 -- 172.21.15.72:0/2566421159 shutdown_connections 2023-10-18T17:59:28.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988da79700 1 --2- 172.21.15.72:0/2566421159 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881513e0 0x7f9888067d30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:28.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988da79700 1 -- 172.21.15.72:0/2566421159 >> 172.21.15.72:0/2566421159 conn(0x7f9888062ba0 msgr2=0x7f98880c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:28.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988da79700 1 -- 172.21.15.72:0/2566421159 shutdown_connections 2023-10-18T17:59:28.057 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.045+0000 7f988da79700 1 -- 172.21.15.72:0/2566421159 wait complete. 2023-10-18T17:59:28.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988da79700 1 Processor -- start 2023-10-18T17:59:28.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988da79700 1 -- start start 2023-10-18T17:59:28.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988da79700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98880630e0 0x7f9888165f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:28.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988da79700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98881513e0 0x7f9888166470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:28.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988da79700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881517c0 0x7f98881561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:28.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988da79700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9888166960 con 0x7f98881517c0 2023-10-18T17:59:28.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988da79700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988815b2c0 con 0x7f98880630e0 2023-10-18T17:59:28.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988da79700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9888152900 con 0x7f98881513e0 2023-10-18T17:59:28.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988d278700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881517c0 0x7f98881561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:28.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988d278700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881517c0 0x7f98881561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45020/0 (socket says 172.21.15.72:45020) 2023-10-18T17:59:28.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988ca77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98880630e0 0x7f9888165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:28.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f987ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98881513e0 0x7f9888166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:28.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988d278700 1 -- 172.21.15.72:0/334730626 learned_addr learned my addr 172.21.15.72:0/334730626 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:28.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988d278700 1 -- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98881513e0 msgr2=0x7f9888166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:28.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988d278700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98881513e0 0x7f9888166470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:28.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988d278700 1 -- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98880630e0 msgr2=0x7f9888165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:28.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988d278700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98880630e0 0x7f9888165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:28.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.049+0000 7f988d278700 1 -- 172.21.15.72:0/334730626 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840092a0 con 0x7f98881517c0 2023-10-18T17:59:28.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.053+0000 7f988d278700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881517c0 0x7f98881561d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f988000c7e0 tx=0x7f9880007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.053+0000 7f987cff9700 1 -- 172.21.15.72:0/334730626 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880005c70 con 0x7f98881517c0 2023-10-18T17:59:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.053+0000 7f987cff9700 1 -- 172.21.15.72:0/334730626 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880005dd0 con 0x7f98881517c0 2023-10-18T17:59:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.053+0000 7f987cff9700 1 -- 172.21.15.72:0/334730626 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880007320 con 0x7f98881517c0 2023-10-18T17:59:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.053+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9888152bc0 con 0x7f98881517c0 2023-10-18T17:59:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.053+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9888153180 con 0x7f98881517c0 2023-10-18T17:59:28.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.053+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9888046f00 con 0x7f98881517c0 2023-10-18T17:59:28.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.053+0000 7f987cff9700 1 -- 172.21.15.72:0/334730626 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9880007710 con 0x7f98881517c0 2023-10-18T17:59:28.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.057+0000 7f987cff9700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f987405a510 0x7f987405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:28.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.057+0000 7f987cff9700 1 -- 172.21.15.72:0/334730626 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f9880074820 con 0x7f98881517c0 2023-10-18T17:59:28.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.057+0000 7f988ca77700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f987405a510 0x7f987405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:28.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.057+0000 7f988ca77700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f987405a510 0x7f987405c9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f988400b3d0 tx=0x7f9884017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:28.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.061+0000 7f987cff9700 1 -- 172.21.15.72:0/334730626 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f9880047ed0 con 0x7f98881517c0 2023-10-18T17:59:28.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.257+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9888165ce0 con 0x7f98881517c0 2023-10-18T17:59:28.261 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.257+0000 7f987cff9700 1 -- 172.21.15.72:0/334730626 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f988004b4f0 con 0x7f98881517c0 2023-10-18T17:59:28.262 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:28.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f987405a510 msgr2=0x7f987405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:28.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f987405a510 0x7f987405c9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f988400b3d0 tx=0x7f9884017000).stop 2023-10-18T17:59:28.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881517c0 msgr2=0x7f98881561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:28.266 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881517c0 0x7f98881561d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f988000c7e0 tx=0x7f9880007f60).stop 2023-10-18T17:59:28.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 shutdown_connections 2023-10-18T17:59:28.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f987405a510 0x7f987405c9d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:28.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98881517c0 0x7f98881561d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:28.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98881513e0 0x7f9888166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:28.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 --2- 172.21.15.72:0/334730626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98880630e0 0x7f9888165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:28.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 >> 172.21.15.72:0/334730626 conn(0x7f9888062ba0 msgr2=0x7f98880656e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:28.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 shutdown_connections 2023-10-18T17:59:28.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:28.261+0000 7f988da79700 1 -- 172.21.15.72:0/334730626 wait complete. 2023-10-18T17:59:28.288 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:31.289 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:31.457 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.453+0000 7f9ef09da700 1 Processor -- start 2023-10-18T17:59:31.458 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.453+0000 7f9ef09da700 1 -- start start 2023-10-18T17:59:31.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.453+0000 7f9ef09da700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0658a0 0x7f9eec065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:31.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.453+0000 7f9ef09da700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9eec0630e0 0x7f9eec14cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:31.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.453+0000 7f9ef09da700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec06a760 con 0x7f9eec069f20 2023-10-18T17:59:31.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.453+0000 7f9ef09da700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eec0c2d40 con 0x7f9eec0630e0 2023-10-18T17:59:31.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.453+0000 7f9ef09da700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec150bf0 con 0x7f9eec0658a0 2023-10-18T17:59:31.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eeb59e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9eec0630e0 0x7f9eec14cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:31.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eeb59e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9eec0630e0 0x7f9eec14cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54354/0 (socket says 172.21.15.72:54354) 2023-10-18T17:59:31.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eeb59e700 1 -- 172.21.15.72:0/35827065 learned_addr learned my addr 172.21.15.72:0/35827065 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:31.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eead9d700 1 --2- 172.21.15.72:0/35827065 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0658a0 0x7f9eec065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:31.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eea59c700 1 --1- 172.21.15.72:0/35827065 >> v1:172.21.15.72:6789/0 conn(0x7f9eec069f20 0x7f9eec0c2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43094/0 (socket says 172.21.15.72:43094) 2023-10-18T17:59:31.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eeb59e700 1 -- 172.21.15.72:0/35827065 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0658a0 msgr2=0x7f9eec065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:31.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eeb59e700 1 --2- 172.21.15.72:0/35827065 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0658a0 0x7f9eec065c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:31.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eeb59e700 1 -- 172.21.15.72:0/35827065 >> v1:172.21.15.72:6789/0 conn(0x7f9eec069f20 legacy=0x7f9eec0c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:31.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eeb59e700 1 -- 172.21.15.72:0/35827065 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec06a480 con 0x7f9eec0630e0 2023-10-18T17:59:31.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9eeb59e700 1 --2- 172.21.15.72:0/35827065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0630e0 0x7f9eec14cda0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f9ee4009f20 tx=0x7f9ee4009840).ready entity=mon.0 client_cookie=93f3743ed8de3a84 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:31.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ee9d9b700 1 -- 172.21.15.72:0/35827065 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee400e070 con 0x7f9eec0630e0 2023-10-18T17:59:31.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ee9d9b700 1 -- 172.21.15.72:0/35827065 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee400ba60 con 0x7f9eec0630e0 2023-10-18T17:59:31.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ee9d9b700 1 -- 172.21.15.72:0/35827065 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee400bde0 con 0x7f9eec0630e0 2023-10-18T17:59:31.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ef09da700 1 -- 172.21.15.72:0/35827065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0630e0 msgr2=0x7f9eec14cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:31.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ef09da700 1 --2- 172.21.15.72:0/35827065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0630e0 0x7f9eec14cda0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f9ee4009f20 tx=0x7f9ee4009840).stop 2023-10-18T17:59:31.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ef09da700 1 -- 172.21.15.72:0/35827065 shutdown_connections 2023-10-18T17:59:31.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ef09da700 1 --2- 172.21.15.72:0/35827065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0630e0 0x7f9eec14cda0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:31.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ef09da700 1 --2- 172.21.15.72:0/35827065 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0658a0 0x7f9eec065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:31.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ef09da700 1 -- 172.21.15.72:0/35827065 >> 172.21.15.72:0/35827065 conn(0x7f9eec062ba0 msgr2=0x7f9eec0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:31.471 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ef09da700 1 -- 172.21.15.72:0/35827065 shutdown_connections 2023-10-18T17:59:31.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.457+0000 7f9ef09da700 1 -- 172.21.15.72:0/35827065 wait complete. 2023-10-18T17:59:31.475 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.461+0000 7f9ef09da700 1 Processor -- start 2023-10-18T17:59:31.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.461+0000 7f9ef09da700 1 -- start start 2023-10-18T17:59:31.476 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ef09da700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0630e0 0x7f9eec0bc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:31.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ef09da700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0658a0 0x7f9eec156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:31.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ef09da700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec069f20 0x7f9eec1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:31.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ef09da700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec1556b0 con 0x7f9eec0658a0 2023-10-18T17:59:31.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ef09da700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0bcec0 con 0x7f9eec069f20 2023-10-18T17:59:31.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ef09da700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0bd480 con 0x7f9eec0630e0 2023-10-18T17:59:31.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eea59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0658a0 0x7f9eec156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:31.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eeb59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec069f20 0x7f9eec1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:31.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eead9d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0630e0 0x7f9eec0bc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:31.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eea59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0658a0 0x7f9eec156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54356/0 (socket says 172.21.15.72:54356) 2023-10-18T17:59:31.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eea59c700 1 -- 172.21.15.72:0/2656484693 learned_addr learned my addr 172.21.15.72:0/2656484693 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:31.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eea59c700 1 -- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0630e0 msgr2=0x7f9eec0bc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:31.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eea59c700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0630e0 0x7f9eec0bc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:31.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eea59c700 1 -- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec069f20 msgr2=0x7f9eec1574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:31.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eea59c700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec069f20 0x7f9eec1574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:31.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eea59c700 1 -- 172.21.15.72:0/2656484693 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee4009ae0 con 0x7f9eec0658a0 2023-10-18T17:59:31.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eeb59e700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec069f20 0x7f9eec1574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:59:31.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9eea59c700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0658a0 0x7f9eec156fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f9ee000c4f0 tx=0x7f9ee0007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:31.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ed2ffd700 1 -- 172.21.15.72:0/2656484693 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000ce00 con 0x7f9eec0658a0 2023-10-18T17:59:31.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ed2ffd700 1 -- 172.21.15.72:0/2656484693 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee00072e0 con 0x7f9eec0658a0 2023-10-18T17:59:31.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ed2ffd700 1 -- 172.21.15.72:0/2656484693 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00056e0 con 0x7f9eec0658a0 2023-10-18T17:59:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec1579c0 con 0x7f9eec0658a0 2023-10-18T17:59:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.465+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec163f50 con 0x7f9eec0658a0 2023-10-18T17:59:31.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.469+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eec0bd610 con 0x7f9eec0658a0 2023-10-18T17:59:31.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.469+0000 7f9ed2ffd700 1 -- 172.21.15.72:0/2656484693 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9ee0005b10 con 0x7f9eec0658a0 2023-10-18T17:59:31.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.473+0000 7f9ed2ffd700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9ed805a510 0x7f9ed805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:31.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.473+0000 7f9ed2ffd700 1 -- 172.21.15.72:0/2656484693 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f9ee0073e60 con 0x7f9eec0658a0 2023-10-18T17:59:31.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.473+0000 7f9eead9d700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9ed805a510 0x7f9ed805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:31.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.473+0000 7f9eead9d700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9ed805a510 0x7f9ed805c9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9edc009b90 tx=0x7f9edc009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:31.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.477+0000 7f9ed2ffd700 1 -- 172.21.15.72:0/2656484693 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f9ee00474b0 con 0x7f9eec0658a0 2023-10-18T17:59:31.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.669+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9eec046f00 con 0x7f9eec0658a0 2023-10-18T17:59:31.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.669+0000 7f9ed2ffd700 1 -- 172.21.15.72:0/2656484693 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f9ee0018070 con 0x7f9eec0658a0 2023-10-18T17:59:31.677 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:31.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9ed805a510 msgr2=0x7f9ed805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:31.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9ed805a510 0x7f9ed805c9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9edc009b90 tx=0x7f9edc009060).stop 2023-10-18T17:59:31.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0658a0 msgr2=0x7f9eec156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:31.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0658a0 0x7f9eec156fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f9ee000c4f0 tx=0x7f9ee0007dc0).stop 2023-10-18T17:59:31.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 shutdown_connections 2023-10-18T17:59:31.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9ed805a510 0x7f9ed805c9d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:31.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec069f20 0x7f9eec1574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:31.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9eec0658a0 0x7f9eec156fe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:31.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 --2- 172.21.15.72:0/2656484693 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9eec0630e0 0x7f9eec0bc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:31.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 >> 172.21.15.72:0/2656484693 conn(0x7f9eec062ba0 msgr2=0x7f9eec14e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:31.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 shutdown_connections 2023-10-18T17:59:31.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:31.673+0000 7f9ef09da700 1 -- 172.21.15.72:0/2656484693 wait complete. 2023-10-18T17:59:31.698 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:32.359 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~61s 2023-10-18T17:59:32.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2023-10-18T17:59:32.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~20s 2023-10-18T17:59:34.699 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:34.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.856+0000 7f51bca1a700 1 Processor -- start 2023-10-18T17:59:34.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.856+0000 7f51bca1a700 1 -- start start 2023-10-18T17:59:34.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51bca1a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f51b814d170 0x7f51b806a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:34.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51bca1a700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f51b8068180 0x7f51b8065700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:34.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51bca1a700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b8068910 con 0x7f51b8067d30 2023-10-18T17:59:34.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51bca1a700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b8150c10 con 0x7f51b8068180 2023-10-18T17:59:34.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51bca1a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b8150780 con 0x7f51b814d170 2023-10-18T17:59:34.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b6d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f51b814d170 0x7f51b806a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:34.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b6d9d700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f51b814d170 0x7f51b806a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:58744/0 (socket says 172.21.15.72:58744) 2023-10-18T17:59:34.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b6d9d700 1 -- 172.21.15.72:0/3334525456 learned_addr learned my addr 172.21.15.72:0/3334525456 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:34.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b759e700 1 --2- 172.21.15.72:0/3334525456 >> v2:172.21.15.72:3300/0 conn(0x7f51b8068180 0x7f51b8065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:34.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b6d9d700 1 -- 172.21.15.72:0/3334525456 >> v2:172.21.15.72:3300/0 conn(0x7f51b8068180 msgr2=0x7f51b8065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:34.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b6d9d700 1 --2- 172.21.15.72:0/3334525456 >> v2:172.21.15.72:3300/0 conn(0x7f51b8068180 0x7f51b8065700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b6d9d700 1 -- 172.21.15.72:0/3334525456 >> v1:172.21.15.72:6789/0 conn(0x7f51b8067d30 legacy=0x7f51b80630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b6d9d700 1 -- 172.21.15.72:0/3334525456 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b8068630 con 0x7f51b814d170 2023-10-18T17:59:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b659c700 1 -- 172.21.15.72:0/3334525456 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523680286 0 0) 0x7f51b8068910 con 0x7f51b8067d30 2023-10-18T17:59:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.860+0000 7f51b6d9d700 1 --2- 172.21.15.72:0/3334525456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b814d170 0x7f51b806a150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f51b8063070 tx=0x7f51a80090f0).ready entity=mon.1 client_cookie=cd10a87c4d0bcc53 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:34.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51b659c700 1 -- 172.21.15.72:0/3334525456 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800b5b0 con 0x7f51b814d170 2023-10-18T17:59:34.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51b659c700 1 -- 172.21.15.72:0/3334525456 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a800b710 con 0x7f51b814d170 2023-10-18T17:59:34.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51b659c700 1 -- 172.21.15.72:0/3334525456 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800ba90 con 0x7f51b814d170 2023-10-18T17:59:34.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51bca1a700 1 -- 172.21.15.72:0/3334525456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b814d170 msgr2=0x7f51b806a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:34.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51bca1a700 1 --2- 172.21.15.72:0/3334525456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b814d170 0x7f51b806a150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f51b8063070 tx=0x7f51a80090f0).stop 2023-10-18T17:59:34.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51bca1a700 1 -- 172.21.15.72:0/3334525456 shutdown_connections 2023-10-18T17:59:34.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51bca1a700 1 --2- 172.21.15.72:0/3334525456 >> v2:172.21.15.72:3300/0 conn(0x7f51b8068180 0x7f51b8065700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:34.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51bca1a700 1 --2- 172.21.15.72:0/3334525456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b814d170 0x7f51b806a150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:34.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51bca1a700 1 -- 172.21.15.72:0/3334525456 >> 172.21.15.72:0/3334525456 conn(0x7f51b8062ba0 msgr2=0x7f51b80c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:34.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51bca1a700 1 -- 172.21.15.72:0/3334525456 shutdown_connections 2023-10-18T17:59:34.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51bca1a700 1 -- 172.21.15.72:0/3334525456 wait complete. 2023-10-18T17:59:34.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.864+0000 7f51bca1a700 1 Processor -- start 2023-10-18T17:59:34.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51bca1a700 1 -- start start 2023-10-18T17:59:34.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51bca1a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51b8067d30 0x7f51b80c0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:34.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51bca1a700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b8068180 0x7f51b8153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:34.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51bca1a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b814d170 0x7f51b814b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:34.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51bca1a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b8153960 con 0x7f51b8067d30 2023-10-18T17:59:34.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51bca1a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81550e0 con 0x7f51b814d170 2023-10-18T17:59:34.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51bca1a700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81514d0 con 0x7f51b8068180 2023-10-18T17:59:34.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b6d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51b8067d30 0x7f51b80c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b6d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51b8067d30 0x7f51b80c0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54382/0 (socket says 172.21.15.72:54382) 2023-10-18T17:59:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b6d9d700 1 -- 172.21.15.72:0/2082942290 learned_addr learned my addr 172.21.15.72:0/2082942290 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b759e700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b814d170 0x7f51b814b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51affff700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b8068180 0x7f51b8153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:34.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b6d9d700 1 -- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b8068180 msgr2=0x7f51b8153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b6d9d700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b8068180 0x7f51b8153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b6d9d700 1 -- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b814d170 msgr2=0x7f51b814b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b6d9d700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b814d170 0x7f51b814b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:34.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b6d9d700 1 -- 172.21.15.72:0/2082942290 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8009370 con 0x7f51b8067d30 2023-10-18T17:59:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51b6d9d700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51b8067d30 0x7f51b80c0730 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f51a8009130 tx=0x7f51a8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51aeffd700 1 -- 172.21.15.72:0/2082942290 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800be50 con 0x7f51b8067d30 2023-10-18T17:59:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51aeffd700 1 -- 172.21.15.72:0/2082942290 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a800b880 con 0x7f51b8067d30 2023-10-18T17:59:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b8161410 con 0x7f51b8067d30 2023-10-18T17:59:34.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51aeffd700 1 -- 172.21.15.72:0/2082942290 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a801b660 con 0x7f51b8067d30 2023-10-18T17:59:34.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.868+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b8153580 con 0x7f51b8067d30 2023-10-18T17:59:34.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.872+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b80c28f0 con 0x7f51b8067d30 2023-10-18T17:59:34.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.872+0000 7f51aeffd700 1 -- 172.21.15.72:0/2082942290 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f51a801b9d0 con 0x7f51b8067d30 2023-10-18T17:59:34.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.872+0000 7f51aeffd700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f519805a540 0x7f519805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:34.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.872+0000 7f51aeffd700 1 -- 172.21.15.72:0/2082942290 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f51a80756b0 con 0x7f51b8067d30 2023-10-18T17:59:34.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.872+0000 7f51affff700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f519805a540 0x7f519805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:34.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.876+0000 7f51affff700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f519805a540 0x7f519805ca00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f51b8150550 tx=0x7f51a0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:34.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:34.880+0000 7f51aeffd700 1 -- 172.21.15.72:0/2082942290 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f51a8048d00 con 0x7f51b8067d30 2023-10-18T17:59:35.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.080+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f51b81518e0 con 0x7f51b8067d30 2023-10-18T17:59:35.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.084+0000 7f51aeffd700 1 -- 172.21.15.72:0/2082942290 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f51a8021020 con 0x7f51b8067d30 2023-10-18T17:59:35.090 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:35.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f519805a540 msgr2=0x7f519805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:35.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f519805a540 0x7f519805ca00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f51b8150550 tx=0x7f51a0009040).stop 2023-10-18T17:59:35.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51b8067d30 msgr2=0x7f51b80c0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:35.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51b8067d30 0x7f51b80c0730 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f51a8009130 tx=0x7f51a8017000).stop 2023-10-18T17:59:35.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 shutdown_connections 2023-10-18T17:59:35.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f519805a540 0x7f519805ca00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:35.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51b814d170 0x7f51b814b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:35.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f51b8068180 0x7f51b8153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:35.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 --2- 172.21.15.72:0/2082942290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51b8067d30 0x7f51b80c0730 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:35.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 >> 172.21.15.72:0/2082942290 conn(0x7f51b8062ba0 msgr2=0x7f51b80c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:35.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 shutdown_connections 2023-10-18T17:59:35.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:35.088+0000 7f51bca1a700 1 -- 172.21.15.72:0/2082942290 wait complete. 2023-10-18T17:59:35.114 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:38.116 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:38.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.268+0000 7fabfbc5b700 1 Processor -- start 2023-10-18T17:59:38.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.268+0000 7fabfbc5b700 1 -- start start 2023-10-18T17:59:38.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabfbc5b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fabf4067ff0 0x7fabf40683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:38.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabfbc5b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4065e30 0x7fabf40bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:38.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabfbc5b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf4066900 con 0x7fabf40bca50 2023-10-18T17:59:38.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabfbc5b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf4068a50 con 0x7fabf4067ff0 2023-10-18T17:59:38.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabfbc5b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40659a0 con 0x7fabf4065e30 2023-10-18T17:59:38.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabfa1f8700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fabf40bca50 0x7fabf4150890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43110/0 (socket says 172.21.15.72:43110) 2023-10-18T17:59:38.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabf91f6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4065e30 0x7fabf40bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:38.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabfa1f8700 1 -- 172.21.15.72:0/2269781017 learned_addr learned my addr 172.21.15.72:0/2269781017 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:38.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabf99f7700 1 --2- 172.21.15.72:0/2269781017 >> v2:172.21.15.72:3300/0 conn(0x7fabf4067ff0 0x7fabf40683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabf89f5700 1 -- 172.21.15.72:0/2269781017 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105771326 0 0) 0x7fabf4066900 con 0x7fabf40bca50 2023-10-18T17:59:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabf99f7700 1 -- 172.21.15.72:0/2269781017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4065e30 msgr2=0x7fabf40bdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabf99f7700 1 --2- 172.21.15.72:0/2269781017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4065e30 0x7fabf40bdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabf99f7700 1 -- 172.21.15.72:0/2269781017 >> v1:172.21.15.72:6789/0 conn(0x7fabf40bca50 legacy=0x7fabf4150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:38.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabf99f7700 1 -- 172.21.15.72:0/2269781017 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4066620 con 0x7fabf4067ff0 2023-10-18T17:59:38.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabf91f6700 1 --2- 172.21.15.72:0/2269781017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4065e30 0x7fabf40bdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:59:38.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.272+0000 7fabf99f7700 1 --2- 172.21.15.72:0/2269781017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf4067ff0 0x7fabf40683d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fabf0009e90 tx=0x7fabf0009780).ready entity=mon.0 client_cookie=c95f327d6c5e2fd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:38.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabf89f5700 1 -- 172.21.15.72:0/2269781017 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf000e070 con 0x7fabf4067ff0 2023-10-18T17:59:38.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabf89f5700 1 -- 172.21.15.72:0/2269781017 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf000bbb0 con 0x7fabf4067ff0 2023-10-18T17:59:38.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabf89f5700 1 -- 172.21.15.72:0/2269781017 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0012410 con 0x7fabf4067ff0 2023-10-18T17:59:38.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2269781017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf4067ff0 msgr2=0x7fabf40683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:38.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabfbc5b700 1 --2- 172.21.15.72:0/2269781017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf4067ff0 0x7fabf40683d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fabf0009e90 tx=0x7fabf0009780).stop 2023-10-18T17:59:38.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2269781017 shutdown_connections 2023-10-18T17:59:38.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabfbc5b700 1 --2- 172.21.15.72:0/2269781017 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4065e30 0x7fabf40bdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:38.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabfbc5b700 1 --2- 172.21.15.72:0/2269781017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf4067ff0 0x7fabf40683d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:38.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2269781017 >> 172.21.15.72:0/2269781017 conn(0x7fabf40620d0 msgr2=0x7fabf40624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:38.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2269781017 shutdown_connections 2023-10-18T17:59:38.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.276+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2269781017 wait complete. 2023-10-18T17:59:38.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfbc5b700 1 Processor -- start 2023-10-18T17:59:38.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfbc5b700 1 -- start start 2023-10-18T17:59:38.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfbc5b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabf4065e30 0x7fabf40bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:38.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfbc5b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4067ff0 0x7fabf4150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:38.287 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfbc5b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf40bca50 0x7fabf40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:38.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfbc5b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf414d6b0 con 0x7fabf40bca50 2023-10-18T17:59:38.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfbc5b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4161540 con 0x7fabf4065e30 2023-10-18T17:59:38.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfbc5b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4160930 con 0x7fabf4067ff0 2023-10-18T17:59:38.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfa1f8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf40bca50 0x7fabf40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:38.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfa1f8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf40bca50 0x7fabf40b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54408/0 (socket says 172.21.15.72:54408) 2023-10-18T17:59:38.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfa1f8700 1 -- 172.21.15.72:0/2375980150 learned_addr learned my addr 172.21.15.72:0/2375980150 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:38.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabf99f7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabf4065e30 0x7fabf40bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:38.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabf91f6700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4067ff0 0x7fabf4150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:38.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfa1f8700 1 -- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4067ff0 msgr2=0x7fabf4150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:38.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfa1f8700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4067ff0 0x7fabf4150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:38.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfa1f8700 1 -- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabf4065e30 msgr2=0x7fabf40bb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:38.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfa1f8700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabf4065e30 0x7fabf40bb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:38.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfa1f8700 1 -- 172.21.15.72:0/2375980150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe40092a0 con 0x7fabf40bca50 2023-10-18T17:59:38.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfa1f8700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf40bca50 0x7fabf40b3b00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fabec00c4c0 tx=0x7fabec007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:38.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabe9ffb700 1 -- 172.21.15.72:0/2375980150 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00ce60 con 0x7fabf40bca50 2023-10-18T17:59:38.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabe9ffb700 1 -- 172.21.15.72:0/2375980150 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec0072e0 con 0x7fabf40bca50 2023-10-18T17:59:38.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf0009a20 con 0x7fabf40bca50 2023-10-18T17:59:38.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.280+0000 7fabe9ffb700 1 -- 172.21.15.72:0/2375980150 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0057a0 con 0x7fabf40bca50 2023-10-18T17:59:38.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.284+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf4160ef0 con 0x7fabf40bca50 2023-10-18T17:59:38.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.284+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf406c190 con 0x7fabf40bca50 2023-10-18T17:59:38.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.284+0000 7fabe9ffb700 1 -- 172.21.15.72:0/2375980150 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fabec005b10 con 0x7fabf40bca50 2023-10-18T17:59:38.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.284+0000 7fabe9ffb700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fabd805a510 0x7fabd805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:38.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.284+0000 7fabe9ffb700 1 -- 172.21.15.72:0/2375980150 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fabec073f90 con 0x7fabf40bca50 2023-10-18T17:59:38.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.284+0000 7fabf99f7700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fabd805a510 0x7fabd805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:38.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.288+0000 7fabf99f7700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fabd805a510 0x7fabd805c9d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fabf0009e90 tx=0x7fabf0009780).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:38.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.292+0000 7fabe9ffb700 1 -- 172.21.15.72:0/2375980150 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fabec0476a0 con 0x7fabf40bca50 2023-10-18T17:59:38.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.480+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fabf414c1e0 con 0x7fabf40bca50 2023-10-18T17:59:38.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.480+0000 7fabe9ffb700 1 -- 172.21.15.72:0/2375980150 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fabec018070 con 0x7fabf40bca50 2023-10-18T17:59:38.488 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:38.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fabd805a510 msgr2=0x7fabd805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:38.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fabd805a510 0x7fabd805c9d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fabf0009e90 tx=0x7fabf0009780).stop 2023-10-18T17:59:38.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf40bca50 msgr2=0x7fabf40b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:38.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf40bca50 0x7fabf40b3b00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fabec00c4c0 tx=0x7fabec007dd0).stop 2023-10-18T17:59:38.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 shutdown_connections 2023-10-18T17:59:38.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fabd805a510 0x7fabd805c9d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:38.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabf40bca50 0x7fabf40b3b00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:38.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fabf4067ff0 0x7fabf4150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:38.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 --2- 172.21.15.72:0/2375980150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabf4065e30 0x7fabf40bb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:38.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 >> 172.21.15.72:0/2375980150 conn(0x7fabf40620d0 msgr2=0x7fabf414a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:38.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 shutdown_connections 2023-10-18T17:59:38.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:38.484+0000 7fabfbc5b700 1 -- 172.21.15.72:0/2375980150 wait complete. 2023-10-18T17:59:38.514 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:38.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~68s 2023-10-18T17:59:38.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2023-10-18T17:59:38.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~27s 2023-10-18T17:59:39.558 INFO:tasks.ceph.mds.b.smithi179.stderr:terminate called after throwing an instance of 'boost::wrapexcept' 2023-10-18T17:59:39.559 INFO:tasks.ceph.mds.b.smithi179.stderr: what(): boost::bad_get: failed value get using boost::get 2023-10-18T17:59:39.559 INFO:tasks.ceph.mds.b.smithi179.stderr:*** Caught signal (Aborted) ** 2023-10-18T17:59:39.559 INFO:tasks.ceph.mds.b.smithi179.stderr: in thread 7f1a9b466700 thread_name:ms_dispatch 2023-10-18T17:59:39.560 INFO:tasks.ceph.mds.b.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:39.560 INFO:tasks.ceph.mds.b.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f1aa06c5420] 2023-10-18T17:59:39.560 INFO:tasks.ceph.mds.b.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:39.561 INFO:tasks.ceph.mds.b.smithi179.stderr: 3: abort() 2023-10-18T17:59:39.561 INFO:tasks.ceph.mds.b.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f1aa0567911] 2023-10-18T17:59:39.561 INFO:tasks.ceph.mds.b.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f1aa057338c] 2023-10-18T17:59:39.561 INFO:tasks.ceph.mds.b.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f1aa05733f7] 2023-10-18T17:59:39.561 INFO:tasks.ceph.mds.b.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f1aa05736a9] 2023-10-18T17:59:39.561 INFO:tasks.ceph.mds.b.smithi179.stderr: 8: ceph-mds(+0x185107) [0x55fb5d363107] 2023-10-18T17:59:39.562 INFO:tasks.ceph.mds.b.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x55fb5d36372d] 2023-10-18T17:59:39.562 INFO:tasks.ceph.mds.b.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55fb5d3d509f] 2023-10-18T17:59:39.562 INFO:tasks.ceph.mds.b.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55fb5d3d6a95] 2023-10-18T17:59:39.562 INFO:tasks.ceph.mds.b.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55fb5d3a1b10] 2023-10-18T17:59:39.563 INFO:tasks.ceph.mds.b.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55fb5d3a5abf] 2023-10-18T17:59:39.563 INFO:tasks.ceph.mds.b.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55fb5d3a6456] 2023-10-18T17:59:39.563 INFO:tasks.ceph.mds.b.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f1aa0bcb710] 2023-10-18T17:59:39.563 INFO:tasks.ceph.mds.b.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f1aa0bc8faf] 2023-10-18T17:59:39.563 INFO:tasks.ceph.mds.b.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1aa0c97b21] 2023-10-18T17:59:39.564 INFO:tasks.ceph.mds.b.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1aa06b9609] 2023-10-18T17:59:39.564 INFO:tasks.ceph.mds.b.smithi179.stderr: 19: clone() 2023-10-18T17:59:39.564 INFO:tasks.ceph.mds.b.smithi179.stderr:2023-10-18T17:59:39.555+0000 7f1a9b466700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:39.564 INFO:tasks.ceph.mds.b.smithi179.stderr: in thread 7f1a9b466700 thread_name:ms_dispatch 2023-10-18T17:59:39.564 INFO:tasks.ceph.mds.b.smithi179.stderr: 2023-10-18T17:59:39.565 INFO:tasks.ceph.mds.b.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:39.565 INFO:tasks.ceph.mds.b.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f1aa06c5420] 2023-10-18T17:59:39.565 INFO:tasks.ceph.mds.b.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:39.565 INFO:tasks.ceph.mds.b.smithi179.stderr: 3: abort() 2023-10-18T17:59:39.565 INFO:tasks.ceph.mds.b.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f1aa0567911] 2023-10-18T17:59:39.565 INFO:tasks.ceph.mds.b.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f1aa057338c] 2023-10-18T17:59:39.566 INFO:tasks.ceph.mds.b.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f1aa05733f7] 2023-10-18T17:59:39.566 INFO:tasks.ceph.mds.b.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f1aa05736a9] 2023-10-18T17:59:39.566 INFO:tasks.ceph.mds.b.smithi179.stderr: 8: ceph-mds(+0x185107) [0x55fb5d363107] 2023-10-18T17:59:39.566 INFO:tasks.ceph.mds.b.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x55fb5d36372d] 2023-10-18T17:59:39.566 INFO:tasks.ceph.mds.b.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55fb5d3d509f] 2023-10-18T17:59:39.567 INFO:tasks.ceph.mds.b.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55fb5d3d6a95] 2023-10-18T17:59:39.567 INFO:tasks.ceph.mds.b.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55fb5d3a1b10] 2023-10-18T17:59:39.567 INFO:tasks.ceph.mds.b.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55fb5d3a5abf] 2023-10-18T17:59:39.567 INFO:tasks.ceph.mds.b.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55fb5d3a6456] 2023-10-18T17:59:39.567 INFO:tasks.ceph.mds.b.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f1aa0bcb710] 2023-10-18T17:59:39.568 INFO:tasks.ceph.mds.b.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f1aa0bc8faf] 2023-10-18T17:59:39.568 INFO:tasks.ceph.mds.b.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1aa0c97b21] 2023-10-18T17:59:39.568 INFO:tasks.ceph.mds.b.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1aa06b9609] 2023-10-18T17:59:39.568 INFO:tasks.ceph.mds.b.smithi179.stderr: 19: clone() 2023-10-18T17:59:39.568 INFO:tasks.ceph.mds.b.smithi179.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:39.569 INFO:tasks.ceph.mds.b.smithi179.stderr: 2023-10-18T17:59:39.569 INFO:tasks.ceph.mds.b.smithi179.stderr: 0> 2023-10-18T17:59:39.555+0000 7f1a9b466700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:39.569 INFO:tasks.ceph.mds.b.smithi179.stderr: in thread 7f1a9b466700 thread_name:ms_dispatch 2023-10-18T17:59:39.570 INFO:tasks.ceph.mds.b.smithi179.stderr: 2023-10-18T17:59:39.570 INFO:tasks.ceph.mds.b.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:39.570 INFO:tasks.ceph.mds.b.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f1aa06c5420] 2023-10-18T17:59:39.570 INFO:tasks.ceph.mds.b.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:39.570 INFO:tasks.ceph.mds.b.smithi179.stderr: 3: abort() 2023-10-18T17:59:39.571 INFO:tasks.ceph.mds.b.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f1aa0567911] 2023-10-18T17:59:39.571 INFO:tasks.ceph.mds.b.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f1aa057338c] 2023-10-18T17:59:39.571 INFO:tasks.ceph.mds.b.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f1aa05733f7] 2023-10-18T17:59:39.571 INFO:tasks.ceph.mds.b.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f1aa05736a9] 2023-10-18T17:59:39.571 INFO:tasks.ceph.mds.b.smithi179.stderr: 8: ceph-mds(+0x185107) [0x55fb5d363107] 2023-10-18T17:59:39.572 INFO:tasks.ceph.mds.b.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x55fb5d36372d] 2023-10-18T17:59:39.572 INFO:tasks.ceph.mds.b.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55fb5d3d509f] 2023-10-18T17:59:39.572 INFO:tasks.ceph.mds.b.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55fb5d3d6a95] 2023-10-18T17:59:39.572 INFO:tasks.ceph.mds.b.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55fb5d3a1b10] 2023-10-18T17:59:39.572 INFO:tasks.ceph.mds.b.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55fb5d3a5abf] 2023-10-18T17:59:39.572 INFO:tasks.ceph.mds.b.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55fb5d3a6456] 2023-10-18T17:59:39.573 INFO:tasks.ceph.mds.b.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f1aa0bcb710] 2023-10-18T17:59:39.573 INFO:tasks.ceph.mds.b.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f1aa0bc8faf] 2023-10-18T17:59:39.573 INFO:tasks.ceph.mds.b.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1aa0c97b21] 2023-10-18T17:59:39.573 INFO:tasks.ceph.mds.b.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1aa06b9609] 2023-10-18T17:59:39.573 INFO:tasks.ceph.mds.b.smithi179.stderr: 19: clone() 2023-10-18T17:59:39.574 INFO:tasks.ceph.mds.b.smithi179.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:39.574 INFO:tasks.ceph.mds.b.smithi179.stderr: 2023-10-18T17:59:39.574 INFO:tasks.ceph.mds.b.smithi179.stderr: 0> 2023-10-18T17:59:39.555+0000 7f1a9b466700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:39.574 INFO:tasks.ceph.mds.b.smithi179.stderr: in thread 7f1a9b466700 thread_name:ms_dispatch 2023-10-18T17:59:39.574 INFO:tasks.ceph.mds.b.smithi179.stderr: 2023-10-18T17:59:39.575 INFO:tasks.ceph.mds.b.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:39.575 INFO:tasks.ceph.mds.b.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f1aa06c5420] 2023-10-18T17:59:39.575 INFO:tasks.ceph.mds.b.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:39.575 INFO:tasks.ceph.mds.b.smithi179.stderr: 3: abort() 2023-10-18T17:59:39.575 INFO:tasks.ceph.mds.b.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f1aa0567911] 2023-10-18T17:59:39.575 INFO:tasks.ceph.mds.b.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f1aa057338c] 2023-10-18T17:59:39.576 INFO:tasks.ceph.mds.b.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f1aa05733f7] 2023-10-18T17:59:39.576 INFO:tasks.ceph.mds.b.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f1aa05736a9] 2023-10-18T17:59:39.576 INFO:tasks.ceph.mds.b.smithi179.stderr: 8: ceph-mds(+0x185107) [0x55fb5d363107] 2023-10-18T17:59:39.576 INFO:tasks.ceph.mds.b.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x55fb5d36372d] 2023-10-18T17:59:39.576 INFO:tasks.ceph.mds.b.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55fb5d3d509f] 2023-10-18T17:59:39.577 INFO:tasks.ceph.mds.b.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55fb5d3d6a95] 2023-10-18T17:59:39.577 INFO:tasks.ceph.mds.b.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55fb5d3a1b10] 2023-10-18T17:59:39.577 INFO:tasks.ceph.mds.b.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55fb5d3a5abf] 2023-10-18T17:59:39.577 INFO:tasks.ceph.mds.b.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55fb5d3a6456] 2023-10-18T17:59:39.577 INFO:tasks.ceph.mds.b.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f1aa0bcb710] 2023-10-18T17:59:39.577 INFO:tasks.ceph.mds.b.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f1aa0bc8faf] 2023-10-18T17:59:39.578 INFO:tasks.ceph.mds.b.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1aa0c97b21] 2023-10-18T17:59:39.578 INFO:tasks.ceph.mds.b.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f1aa06b9609] 2023-10-18T17:59:39.578 INFO:tasks.ceph.mds.b.smithi179.stderr: 19: clone() 2023-10-18T17:59:39.578 INFO:tasks.ceph.mds.b.smithi179.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:39.578 INFO:tasks.ceph.mds.b.smithi179.stderr: 2023-10-18T17:59:39.817 INFO:tasks.ceph.mds.b.smithi179.stderr:daemon-helper: command crashed with signal 6 2023-10-18T17:59:41.516 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:41.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.676+0000 7f3dd4d69700 1 Processor -- start 2023-10-18T17:59:41.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.676+0000 7f3dd4d69700 1 -- start start 2023-10-18T17:59:41.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dd4d69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd00b9690 0x7f3dd00b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:41.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dd4d69700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3dd00bca50 0x7f3dd014dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:41.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dd4d69700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd00bc480 con 0x7f3dd0065620 2023-10-18T17:59:41.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dd4d69700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd0065190 con 0x7f3dd00bca50 2023-10-18T17:59:41.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dd4d69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00bb920 con 0x7f3dd00b9690 2023-10-18T17:59:41.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dce59c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd00b9690 0x7f3dd00b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:41.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcdd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3dd00bca50 0x7f3dd014dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:41.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcdd9b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f3dd00bca50 0x7f3dd014dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:47318/0 (socket says 172.21.15.72:47318) 2023-10-18T17:59:41.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcdd9b700 1 -- 172.21.15.72:0/1787590027 learned_addr learned my addr 172.21.15.72:0/1787590027 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:41.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcdd9b700 1 -- 172.21.15.72:0/1787590027 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd00b9690 msgr2=0x7f3dd00b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:41.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcdd9b700 1 --2- 172.21.15.72:0/1787590027 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd00b9690 0x7f3dd00b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:41.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcdd9b700 1 -- 172.21.15.72:0/1787590027 >> v1:172.21.15.72:6789/0 conn(0x7f3dd0065620 legacy=0x7f3dd0065a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:41.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcd59a700 1 -- 172.21.15.72:0/1787590027 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277142689 0 0) 0x7f3dd00bc480 con 0x7f3dd0065620 2023-10-18T17:59:41.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcdd9b700 1 -- 172.21.15.72:0/1787590027 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00bbbe0 con 0x7f3dd00bca50 2023-10-18T17:59:41.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcdd9b700 1 --2- 172.21.15.72:0/1787590027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00bca50 0x7f3dd014dd50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f3dc4006fd0 tx=0x7f3dc40090f0).ready entity=mon.1 client_cookie=1d0929c495064162 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:41.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcd59a700 1 -- 172.21.15.72:0/1787590027 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400b580 con 0x7f3dd00bca50 2023-10-18T17:59:41.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcd59a700 1 -- 172.21.15.72:0/1787590027 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400b6e0 con 0x7f3dd00bca50 2023-10-18T17:59:41.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.680+0000 7f3dcd59a700 1 -- 172.21.15.72:0/1787590027 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400ba60 con 0x7f3dd00bca50 2023-10-18T17:59:41.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.684+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1787590027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00bca50 msgr2=0x7f3dd014dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:41.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.684+0000 7f3dd4d69700 1 --2- 172.21.15.72:0/1787590027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00bca50 0x7f3dd014dd50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f3dc4006fd0 tx=0x7f3dc40090f0).stop 2023-10-18T17:59:41.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.684+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1787590027 shutdown_connections 2023-10-18T17:59:41.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.684+0000 7f3dd4d69700 1 --2- 172.21.15.72:0/1787590027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00bca50 0x7f3dd014dd50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:41.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.684+0000 7f3dd4d69700 1 --2- 172.21.15.72:0/1787590027 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd00b9690 0x7f3dd00b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:41.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.684+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1787590027 >> 172.21.15.72:0/1787590027 conn(0x7f3dd00620d0 msgr2=0x7f3dd00624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:41.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.684+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1787590027 shutdown_connections 2023-10-18T17:59:41.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.684+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1787590027 wait complete. 2023-10-18T17:59:41.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.684+0000 7f3dd4d69700 1 Processor -- start 2023-10-18T17:59:41.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dd4d69700 1 -- start start 2023-10-18T17:59:41.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dd4d69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd0065620 0x7f3dd014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:41.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dd4d69700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00b9690 0x7f3dd01459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:41.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dd4d69700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dd014d520 0x7f3dd0145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:41.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dd4d69700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd01463d0 con 0x7f3dd014d520 2023-10-18T17:59:41.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dd4d69700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0068440 con 0x7f3dd00b9690 2023-10-18T17:59:41.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dd4d69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00686e0 con 0x7f3dd0065620 2023-10-18T17:59:41.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dcdd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00b9690 0x7f3dd01459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:41.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dce59c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd0065620 0x7f3dd014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:41.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dcdd9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00b9690 0x7f3dd01459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:47326/0 (socket says 172.21.15.72:47326) 2023-10-18T17:59:41.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dcdd9b700 1 -- 172.21.15.72:0/1654273697 learned_addr learned my addr 172.21.15.72:0/1654273697 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:41.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dced9d700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dd014d520 0x7f3dd0145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:41.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dce59c700 1 -- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00b9690 msgr2=0x7f3dd01459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:41.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dce59c700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00b9690 0x7f3dd01459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:41.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dce59c700 1 -- 172.21.15.72:0/1654273697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dd014d520 msgr2=0x7f3dd0145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:41.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dce59c700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dd014d520 0x7f3dd0145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:41.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dce59c700 1 -- 172.21.15.72:0/1654273697 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc4009370 con 0x7f3dd0065620 2023-10-18T17:59:41.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dce59c700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd0065620 0x7f3dd014d030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3dc000e930 tx=0x7f3dc0009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:41.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dbe7fc700 1 -- 172.21.15.72:0/1654273697 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0009450 con 0x7f3dd0065620 2023-10-18T17:59:41.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.688+0000 7f3dbe7fc700 1 -- 172.21.15.72:0/1654273697 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc0012600 con 0x7f3dd0065620 2023-10-18T17:59:41.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.692+0000 7f3dbe7fc700 1 -- 172.21.15.72:0/1654273697 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0012980 con 0x7f3dd0065620 2023-10-18T17:59:41.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.692+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd00c2730 con 0x7f3dd0065620 2023-10-18T17:59:41.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.692+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd00c2cf0 con 0x7f3dd0065620 2023-10-18T17:59:41.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.692+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd006c190 con 0x7f3dd0065620 2023-10-18T17:59:41.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.692+0000 7f3dbe7fc700 1 -- 172.21.15.72:0/1654273697 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3dc001b400 con 0x7f3dd0065620 2023-10-18T17:59:41.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.692+0000 7f3dbe7fc700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3db005a4e0 0x7f3db005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:41.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.696+0000 7f3dbe7fc700 1 -- 172.21.15.72:0/1654273697 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6408+0+0 (secure 0 0 0) 0x7f3dc0074cc0 con 0x7f3dd0065620 2023-10-18T17:59:41.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.696+0000 7f3dcdd9b700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3db005a4e0 0x7f3db005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:41.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.696+0000 7f3dcdd9b700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3db005a4e0 0x7f3db005c9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f3dd00bf900 tx=0x7f3dc4017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:41.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.704+0000 7f3dbe7fc700 1 -- 172.21.15.72:0/1654273697 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f3dc00483d0 con 0x7f3dd0065620 2023-10-18T17:59:41.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.900+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3dd00bbec0 con 0x7f3dd0065620 2023-10-18T17:59:41.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.900+0000 7f3dbe7fc700 1 -- 172.21.15.72:0/1654273697 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f3dc004b9f0 con 0x7f3dd0065620 2023-10-18T17:59:41.907 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:41.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3db005a4e0 msgr2=0x7f3db005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:41.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3db005a4e0 0x7f3db005c9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f3dd00bf900 tx=0x7f3dc4017000).stop 2023-10-18T17:59:41.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd0065620 msgr2=0x7f3dd014d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:41.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd0065620 0x7f3dd014d030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3dc000e930 tx=0x7f3dc0009060).stop 2023-10-18T17:59:41.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 shutdown_connections 2023-10-18T17:59:41.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3db005a4e0 0x7f3db005c9a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:41.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3dd014d520 0x7f3dd0145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:41.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3dd00b9690 0x7f3dd01459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:41.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 --2- 172.21.15.72:0/1654273697 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3dd0065620 0x7f3dd014d030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:41.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 >> 172.21.15.72:0/1654273697 conn(0x7f3dd00620d0 msgr2=0x7f3dd014f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:41.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 shutdown_connections 2023-10-18T17:59:41.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:41.904+0000 7f3dd4d69700 1 -- 172.21.15.72:0/1654273697 wait complete. 2023-10-18T17:59:41.931 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:44.933 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:45.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.096+0000 7ff9fc169700 1 Processor -- start 2023-10-18T17:59:45.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.096+0000 7ff9fc169700 1 -- start start 2023-10-18T17:59:45.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.096+0000 7ff9fc169700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff9f40b59c0 0x7ff9f40bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:45.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.096+0000 7ff9fc169700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff9f4065b90 0x7ff9f4066040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:45.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.096+0000 7ff9fc169700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f4154250 con 0x7ff9f40bcb40 2023-10-18T17:59:45.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.096+0000 7ff9fc169700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f4066530 con 0x7ff9f40b59c0 2023-10-18T17:59:45.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.096+0000 7ff9fc169700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f41544f0 con 0x7ff9f4065b90 2023-10-18T17:59:45.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.096+0000 7ff9fa706700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff9f4065b90 0x7ff9f4066040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:45.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.096+0000 7ff9f9704700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff9f40b59c0 0x7ff9f40bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:45.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fa706700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff9f4065b90 0x7ff9f4066040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:47338/0 (socket says 172.21.15.72:47338) 2023-10-18T17:59:45.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fa706700 1 -- 172.21.15.72:0/3633432930 learned_addr learned my addr 172.21.15.72:0/3633432930 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:45.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fa706700 1 -- 172.21.15.72:0/3633432930 >> v2:172.21.15.72:3300/0 conn(0x7ff9f40b59c0 msgr2=0x7ff9f40bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:45.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fa706700 1 --2- 172.21.15.72:0/3633432930 >> v2:172.21.15.72:3300/0 conn(0x7ff9f40b59c0 0x7ff9f40bcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:45.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fa706700 1 -- 172.21.15.72:0/3633432930 >> v1:172.21.15.179:6789/0 conn(0x7ff9f40bcb40 legacy=0x7ff9f40b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:45.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fa706700 1 -- 172.21.15.72:0/3633432930 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f41547d0 con 0x7ff9f4065b90 2023-10-18T17:59:45.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fa706700 1 --2- 172.21.15.72:0/3633432930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9f4065b90 0x7ff9f4066040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff9f4154140 tx=0x7ff9ec009840).ready entity=mon.1 client_cookie=8df6c3764f5c80eb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:45.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9f8f03700 1 -- 172.21.15.72:0/3633432930 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00e070 con 0x7ff9f4065b90 2023-10-18T17:59:45.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9f8f03700 1 -- 172.21.15.72:0/3633432930 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec00ba20 con 0x7ff9f4065b90 2023-10-18T17:59:45.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9f8f03700 1 -- 172.21.15.72:0/3633432930 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00bda0 con 0x7ff9f4065b90 2023-10-18T17:59:45.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fc169700 1 -- 172.21.15.72:0/3633432930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9f4065b90 msgr2=0x7ff9f4066040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:45.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fc169700 1 --2- 172.21.15.72:0/3633432930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9f4065b90 0x7ff9f4066040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff9f4154140 tx=0x7ff9ec009840).stop 2023-10-18T17:59:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fc169700 1 -- 172.21.15.72:0/3633432930 shutdown_connections 2023-10-18T17:59:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fc169700 1 --2- 172.21.15.72:0/3633432930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9f4065b90 0x7ff9f4066040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fc169700 1 --2- 172.21.15.72:0/3633432930 >> v2:172.21.15.72:3300/0 conn(0x7ff9f40b59c0 0x7ff9f40bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fc169700 1 -- 172.21.15.72:0/3633432930 >> 172.21.15.72:0/3633432930 conn(0x7ff9f405dc20 msgr2=0x7ff9f405e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:45.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fc169700 1 -- 172.21.15.72:0/3633432930 shutdown_connections 2023-10-18T17:59:45.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.100+0000 7ff9fc169700 1 -- 172.21.15.72:0/3633432930 wait complete. 2023-10-18T17:59:45.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fc169700 1 Processor -- start 2023-10-18T17:59:45.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fc169700 1 -- start start 2023-10-18T17:59:45.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fc169700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9f4065b90 0x7ff9f414c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:45.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fc169700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9f40b59c0 0x7ff9f4164a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:45.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fc169700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9f40bcb40 0x7ff9f40bb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:45.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fc169700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f414fd50 con 0x7ff9f40bcb40 2023-10-18T17:59:45.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fc169700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f414f7e0 con 0x7ff9f4065b90 2023-10-18T17:59:45.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fc169700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f41597b0 con 0x7ff9f40b59c0 2023-10-18T17:59:45.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fa706700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9f40bcb40 0x7ff9f40bb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:45.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fa706700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9f40bcb40 0x7ff9f40bb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45746/0 (socket says 172.21.15.72:45746) 2023-10-18T17:59:45.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fa706700 1 -- 172.21.15.72:0/886081348 learned_addr learned my addr 172.21.15.72:0/886081348 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:45.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9f9f05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9f4065b90 0x7ff9f414c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:45.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9f9704700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9f40b59c0 0x7ff9f4164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:45.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fa706700 1 -- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9f40b59c0 msgr2=0x7ff9f4164a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:45.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fa706700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9f40b59c0 0x7ff9f4164a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:45.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fa706700 1 -- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9f4065b90 msgr2=0x7ff9f414c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:45.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fa706700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9f4065b90 0x7ff9f414c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:45.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fa706700 1 -- 172.21.15.72:0/886081348 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec009ae0 con 0x7ff9f40bcb40 2023-10-18T17:59:45.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fa706700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9f40bcb40 0x7ff9f40bb2d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ff9ec009540 tx=0x7ff9f4155070).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:45.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9e9ffb700 1 -- 172.21.15.72:0/886081348 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec0092e0 con 0x7ff9f40bcb40 2023-10-18T17:59:45.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9e9ffb700 1 -- 172.21.15.72:0/886081348 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec01d410 con 0x7ff9f40bcb40 2023-10-18T17:59:45.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f4164270 con 0x7ff9f40bcb40 2023-10-18T17:59:45.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.104+0000 7ff9e9ffb700 1 -- 172.21.15.72:0/886081348 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec01d7b0 con 0x7ff9f40bcb40 2023-10-18T17:59:45.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.108+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f41594d0 con 0x7ff9f40bcb40 2023-10-18T17:59:45.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.108+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f41640d0 con 0x7ff9f40bcb40 2023-10-18T17:59:45.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.108+0000 7ff9e9ffb700 1 -- 172.21.15.72:0/886081348 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff9ec01db20 con 0x7ff9f40bcb40 2023-10-18T17:59:45.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.112+0000 7ff9e9ffb700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff9d805a510 0x7ff9d805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:45.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.112+0000 7ff9e9ffb700 1 -- 172.21.15.72:0/886081348 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6408+0+0 (secure 0 0 0) 0x7ff9ec076450 con 0x7ff9f40bcb40 2023-10-18T17:59:45.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.112+0000 7ff9f9f05700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff9d805a510 0x7ff9d805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:45.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.112+0000 7ff9f9f05700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff9d805a510 0x7ff9d805c9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff9f0003860 tx=0x7ff9f0007d40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:45.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.116+0000 7ff9e9ffb700 1 -- 172.21.15.72:0/886081348 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ff9f41640d0 con 0x7ff9f40bcb40 2023-10-18T17:59:45.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.312+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff9f41640d0 con 0x7ff9f40bcb40 2023-10-18T17:59:45.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.312+0000 7ff9e9ffb700 1 -- 172.21.15.72:0/886081348 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7ff9f41640d0 con 0x7ff9f40bcb40 2023-10-18T17:59:45.318 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:45.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff9d805a510 msgr2=0x7ff9d805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:45.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff9d805a510 0x7ff9d805c9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff9f0003860 tx=0x7ff9f0007d40).stop 2023-10-18T17:59:45.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9f40bcb40 msgr2=0x7ff9f40bb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:45.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9f40bcb40 0x7ff9f40bb2d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ff9ec009540 tx=0x7ff9f4155070).stop 2023-10-18T17:59:45.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 shutdown_connections 2023-10-18T17:59:45.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff9d805a510 0x7ff9d805c9d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:45.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9f40bcb40 0x7ff9f40bb2d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:45.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff9f40b59c0 0x7ff9f4164a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:45.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 --2- 172.21.15.72:0/886081348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9f4065b90 0x7ff9f414c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:45.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 >> 172.21.15.72:0/886081348 conn(0x7ff9f405dc20 msgr2=0x7ff9f4152800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:45.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 shutdown_connections 2023-10-18T17:59:45.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:45.316+0000 7ff9fc169700 1 -- 172.21.15.72:0/886081348 wait complete. 2023-10-18T17:59:45.344 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:45.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~74s 2023-10-18T17:59:45.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~20s 2023-10-18T17:59:45.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-18T17:59:45.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~33s 2023-10-18T17:59:48.346 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:48.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.504+0000 7f7606b07700 1 Processor -- start 2023-10-18T17:59:48.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.504+0000 7f7606b07700 1 -- start start 2023-10-18T17:59:48.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7606b07700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f76001513e0 0x7f7600067d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:48.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7606b07700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7600154b00 con 0x7f76000630e0 2023-10-18T17:59:48.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7606b07700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760006a750 con 0x7f76001517c0 2023-10-18T17:59:48.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7606b07700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7600069380 con 0x7f76001513e0 2023-10-18T17:59:48.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7606306700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f76000630e0 0x7f7600065700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:40790/0 (socket says 172.21.15.72:40790) 2023-10-18T17:59:48.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7606306700 1 -- 172.21.15.72:0/2662238737 learned_addr learned my addr 172.21.15.72:0/2662238737 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:48.515 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7605b05700 1 --2- 172.21.15.72:0/2662238737 >> v2:172.21.15.179:3300/0 conn(0x7f76001513e0 0x7f7600067d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:48.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7604b03700 1 -- 172.21.15.72:0/2662238737 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901439319 0 0) 0x7f7600154b00 con 0x7f76000630e0 2023-10-18T17:59:48.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7604b03700 1 -- 172.21.15.72:0/2662238737 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75ec0033f0 con 0x7f76000630e0 2023-10-18T17:59:48.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7605b05700 1 -- 172.21.15.72:0/2662238737 >> v1:172.21.15.179:6789/0 conn(0x7f76001517c0 legacy=0x7f760006a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:48.516 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7605b05700 1 -- 172.21.15.72:0/2662238737 >> v1:172.21.15.72:6789/0 conn(0x7f76000630e0 legacy=0x7f7600065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:48.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7605b05700 1 -- 172.21.15.72:0/2662238737 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7600154f90 con 0x7f76001513e0 2023-10-18T17:59:48.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7605b05700 1 --2- 172.21.15.72:0/2662238737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76001513e0 0x7f7600067d30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f75fc009760 tx=0x7f75fc009060).ready entity=mon.1 client_cookie=feb7c7472c1d0895 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:48.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7604b03700 1 -- 172.21.15.72:0/2662238737 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc00b590 con 0x7f76001513e0 2023-10-18T17:59:48.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7604b03700 1 -- 172.21.15.72:0/2662238737 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75fc00b6f0 con 0x7f76001513e0 2023-10-18T17:59:48.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.508+0000 7f7604b03700 1 -- 172.21.15.72:0/2662238737 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc00ba70 con 0x7f76001513e0 2023-10-18T17:59:48.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.512+0000 7f7606b07700 1 -- 172.21.15.72:0/2662238737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76001513e0 msgr2=0x7f7600067d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:48.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.512+0000 7f7606b07700 1 --2- 172.21.15.72:0/2662238737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76001513e0 0x7f7600067d30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f75fc009760 tx=0x7f75fc009060).stop 2023-10-18T17:59:48.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.512+0000 7f7606b07700 1 -- 172.21.15.72:0/2662238737 shutdown_connections 2023-10-18T17:59:48.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.512+0000 7f7606b07700 1 --2- 172.21.15.72:0/2662238737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76001513e0 0x7f7600067d30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:48.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.512+0000 7f7606b07700 1 -- 172.21.15.72:0/2662238737 >> 172.21.15.72:0/2662238737 conn(0x7f7600062ba0 msgr2=0x7f76000c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:48.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.512+0000 7f7606b07700 1 -- 172.21.15.72:0/2662238737 shutdown_connections 2023-10-18T17:59:48.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.512+0000 7f7606b07700 1 -- 172.21.15.72:0/2662238737 wait complete. 2023-10-18T17:59:48.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.512+0000 7f7606b07700 1 Processor -- start 2023-10-18T17:59:48.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7606b07700 1 -- start start 2023-10-18T17:59:48.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7606b07700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76000630e0 0x7f7600156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:48.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7606b07700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76001513e0 0x7f76001534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:48.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7606b07700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76001517c0 0x7f7600158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:48.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7606b07700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76001571e0 con 0x7f76001513e0 2023-10-18T17:59:48.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7606b07700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76001562e0 con 0x7f76000630e0 2023-10-18T17:59:48.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7606b07700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760015c450 con 0x7f76001517c0 2023-10-18T17:59:48.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605b05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76000630e0 0x7f7600156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:48.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7606306700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76001517c0 0x7f7600158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:48.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605b05700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76000630e0 0x7f7600156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:47358/0 (socket says 172.21.15.72:47358) 2023-10-18T17:59:48.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605b05700 1 -- 172.21.15.72:0/733894105 learned_addr learned my addr 172.21.15.72:0/733894105 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:48.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605304700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76001513e0 0x7f76001534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:48.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605b05700 1 -- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76001517c0 msgr2=0x7f7600158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:48.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605b05700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76001517c0 0x7f7600158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:48.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605b05700 1 -- 172.21.15.72:0/733894105 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76001513e0 msgr2=0x7f76001534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:48.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605b05700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76001513e0 0x7f76001534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:48.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605b05700 1 -- 172.21.15.72:0/733894105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc0092a0 con 0x7f76000630e0 2023-10-18T17:59:48.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.516+0000 7f7605b05700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76000630e0 0x7f7600156ba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f7600067180 tx=0x7f75fc017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:48.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.520+0000 7f75f5ffb700 1 -- 172.21.15.72:0/733894105 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc00b590 con 0x7f76000630e0 2023-10-18T17:59:48.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.520+0000 7f75f5ffb700 1 -- 172.21.15.72:0/733894105 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75fc00b860 con 0x7f76000630e0 2023-10-18T17:59:48.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.520+0000 7f75f5ffb700 1 -- 172.21.15.72:0/733894105 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75fc01b660 con 0x7f76000630e0 2023-10-18T17:59:48.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.520+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7600155660 con 0x7f76000630e0 2023-10-18T17:59:48.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.520+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7600155c20 con 0x7f76000630e0 2023-10-18T17:59:48.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.520+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7600046f00 con 0x7f76000630e0 2023-10-18T17:59:48.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.524+0000 7f75f5ffb700 1 -- 172.21.15.72:0/733894105 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f75fc01ba10 con 0x7f76000630e0 2023-10-18T17:59:48.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.524+0000 7f75f5ffb700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f75e405a540 0x7f75e405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:48.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.524+0000 7f75f5ffb700 1 -- 172.21.15.72:0/733894105 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6408+0+0 (secure 0 0 0) 0x7f75fc076170 con 0x7f76000630e0 2023-10-18T17:59:48.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.524+0000 7f7605304700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f75e405a540 0x7f75e405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:48.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.524+0000 7f7605304700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f75e405a540 0x7f75e405ca00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f7600066170 tx=0x7f75f0007d40).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:48.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.536+0000 7f75f5ffb700 1 -- 172.21.15.72:0/733894105 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f75fc049770 con 0x7f76000630e0 2023-10-18T17:59:48.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.736+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7600068570 con 0x7f76000630e0 2023-10-18T17:59:48.741 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.736+0000 7f75f5ffb700 1 -- 172.21.15.72:0/733894105 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f75fc04cd90 con 0x7f76000630e0 2023-10-18T17:59:48.742 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:48.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f75e405a540 msgr2=0x7f75e405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:48.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f75e405a540 0x7f75e405ca00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f7600066170 tx=0x7f75f0007d40).stop 2023-10-18T17:59:48.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76000630e0 msgr2=0x7f7600156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:48.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76000630e0 0x7f7600156ba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f7600067180 tx=0x7f75fc017000).stop 2023-10-18T17:59:48.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 shutdown_connections 2023-10-18T17:59:48.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f75e405a540 0x7f75e405ca00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:48.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f76001517c0 0x7f7600158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:48.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76001513e0 0x7f76001534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:48.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 --2- 172.21.15.72:0/733894105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76000630e0 0x7f7600156ba0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:48.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 >> 172.21.15.72:0/733894105 conn(0x7f7600062ba0 msgr2=0x7f76000c0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:48.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 shutdown_connections 2023-10-18T17:59:48.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:48.740+0000 7f7606b07700 1 -- 172.21.15.72:0/733894105 wait complete. 2023-10-18T17:59:48.767 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:51.768 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:51.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.924+0000 7fc34bfdb700 1 Processor -- start 2023-10-18T17:59:51.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34bfdb700 1 -- start start 2023-10-18T17:59:51.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34bfdb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc3440b9690 0x7fc3440b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:51.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34bfdb700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc344065d30 0x7fc34414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:51.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34bfdb700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3440bc480 con 0x7fc3440bca50 2023-10-18T17:59:51.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34bfdb700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc344065190 con 0x7fc344065d30 2023-10-18T17:59:51.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34bfdb700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3440bb920 con 0x7fc3440b9690 2023-10-18T17:59:51.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34a578700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc344065d30 0x7fc34414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:51.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34a578700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc344065d30 0x7fc34414dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35058/0 (socket says 172.21.15.72:35058) 2023-10-18T17:59:51.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34a578700 1 -- 172.21.15.72:0/213732012 learned_addr learned my addr 172.21.15.72:0/213732012 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:51.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc349d77700 1 --2- 172.21.15.72:0/213732012 >> v2:172.21.15.179:3300/0 conn(0x7fc3440b9690 0x7fc3440b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:51.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34a578700 1 -- 172.21.15.72:0/213732012 >> v2:172.21.15.179:3300/0 conn(0x7fc3440b9690 msgr2=0x7fc3440b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:51.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34a578700 1 --2- 172.21.15.72:0/213732012 >> v2:172.21.15.179:3300/0 conn(0x7fc3440b9690 0x7fc3440b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:51.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34a578700 1 -- 172.21.15.72:0/213732012 >> v1:172.21.15.179:6789/0 conn(0x7fc3440bca50 legacy=0x7fc344065620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:51.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34a578700 1 -- 172.21.15.72:0/213732012 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3440bbbe0 con 0x7fc344065d30 2023-10-18T17:59:51.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc34a578700 1 --2- 172.21.15.72:0/213732012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc344065d30 0x7fc34414dd50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fc344064ac0 tx=0x7fc33c009840).ready entity=mon.0 client_cookie=e22bf49edee58b72 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:51.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc348d75700 1 -- 172.21.15.72:0/213732012 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00e070 con 0x7fc344065d30 2023-10-18T17:59:51.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc348d75700 1 -- 172.21.15.72:0/213732012 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c00ba20 con 0x7fc344065d30 2023-10-18T17:59:51.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.928+0000 7fc348d75700 1 -- 172.21.15.72:0/213732012 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00bda0 con 0x7fc344065d30 2023-10-18T17:59:51.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.932+0000 7fc34bfdb700 1 -- 172.21.15.72:0/213732012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc344065d30 msgr2=0x7fc34414dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:51.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.932+0000 7fc34bfdb700 1 --2- 172.21.15.72:0/213732012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc344065d30 0x7fc34414dd50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fc344064ac0 tx=0x7fc33c009840).stop 2023-10-18T17:59:51.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.932+0000 7fc34bfdb700 1 -- 172.21.15.72:0/213732012 shutdown_connections 2023-10-18T17:59:51.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.932+0000 7fc34bfdb700 1 --2- 172.21.15.72:0/213732012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc344065d30 0x7fc34414dd50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:51.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.932+0000 7fc34bfdb700 1 --2- 172.21.15.72:0/213732012 >> v2:172.21.15.179:3300/0 conn(0x7fc3440b9690 0x7fc3440b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:51.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.932+0000 7fc34bfdb700 1 -- 172.21.15.72:0/213732012 >> 172.21.15.72:0/213732012 conn(0x7fc3440620d0 msgr2=0x7fc3440624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:51.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.932+0000 7fc34bfdb700 1 -- 172.21.15.72:0/213732012 shutdown_connections 2023-10-18T17:59:51.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.932+0000 7fc34bfdb700 1 -- 172.21.15.72:0/213732012 wait complete. 2023-10-18T17:59:51.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.932+0000 7fc34bfdb700 1 Processor -- start 2023-10-18T17:59:51.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc34bfdb700 1 -- start start 2023-10-18T17:59:51.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc34bfdb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc344065d30 0x7fc3440b8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:51.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc34bfdb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3440b9690 0x7fc3440680e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:51.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc34bfdb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3440b8bf0 0x7fc3440685d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:51.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc34bfdb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344143390 con 0x7fc3440b9690 2023-10-18T17:59:51.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc34bfdb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344143930 con 0x7fc3440b8bf0 2023-10-18T17:59:51.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc34bfdb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc344068ac0 con 0x7fc344065d30 2023-10-18T17:59:51.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349576700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3440b9690 0x7fc3440680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:51.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349d77700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc344065d30 0x7fc3440b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:51.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349d77700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc344065d30 0x7fc3440b8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:60948/0 (socket says 172.21.15.72:60948) 2023-10-18T17:59:51.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349d77700 1 -- 172.21.15.72:0/503798562 learned_addr learned my addr 172.21.15.72:0/503798562 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:51.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc34a578700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3440b8bf0 0x7fc3440685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:51.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349576700 1 -- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc344065d30 msgr2=0x7fc3440b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:51.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349576700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc344065d30 0x7fc3440b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:51.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349576700 1 -- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3440b8bf0 msgr2=0x7fc3440685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:51.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349576700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3440b8bf0 0x7fc3440685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:51.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349576700 1 -- 172.21.15.72:0/503798562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c009ae0 con 0x7fc3440b9690 2023-10-18T17:59:51.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc349576700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3440b9690 0x7fc3440680e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fc33400c440 tx=0x7fc334007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:51.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.936+0000 7fc339ffb700 1 -- 172.21.15.72:0/503798562 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334007020 con 0x7fc3440b9690 2023-10-18T17:59:51.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.940+0000 7fc339ffb700 1 -- 172.21.15.72:0/503798562 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc334007180 con 0x7fc3440b9690 2023-10-18T17:59:51.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.940+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34414d000 con 0x7fc3440b9690 2023-10-18T17:59:51.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.940+0000 7fc339ffb700 1 -- 172.21.15.72:0/503798562 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334007520 con 0x7fc3440b9690 2023-10-18T17:59:51.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.940+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3441435c0 con 0x7fc3440b9690 2023-10-18T17:59:51.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.940+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34414d4a0 con 0x7fc3440b9690 2023-10-18T17:59:51.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.940+0000 7fc339ffb700 1 -- 172.21.15.72:0/503798562 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc3340068b0 con 0x7fc3440b9690 2023-10-18T17:59:51.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.944+0000 7fc339ffb700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc32805a510 0x7fc32805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:51.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.944+0000 7fc339ffb700 1 -- 172.21.15.72:0/503798562 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6408+0+0 (secure 0 0 0) 0x7fc334073c10 con 0x7fc3440b9690 2023-10-18T17:59:51.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.944+0000 7fc349d77700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc32805a510 0x7fc32805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:51.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.944+0000 7fc349d77700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc32805a510 0x7fc32805c9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc340009cf0 tx=0x7fc340009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:51.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:51.948+0000 7fc339ffb700 1 -- 172.21.15.72:0/503798562 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fc334047210 con 0x7fc3440b9690 2023-10-18T17:59:51.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~81s 2023-10-18T17:59:51.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~26s 2023-10-18T17:59:51.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-10-18T17:59:51.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~40s 2023-10-18T17:59:52.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.136+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc344152140 con 0x7fc3440b9690 2023-10-18T17:59:52.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.136+0000 7fc339ffb700 1 -- 172.21.15.72:0/503798562 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fc33404a830 con 0x7fc3440b9690 2023-10-18T17:59:52.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:52.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc32805a510 msgr2=0x7fc32805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:52.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc32805a510 0x7fc32805c9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fc340009cf0 tx=0x7fc340009060).stop 2023-10-18T17:59:52.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3440b9690 msgr2=0x7fc3440680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:52.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3440b9690 0x7fc3440680e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fc33400c440 tx=0x7fc334007d40).stop 2023-10-18T17:59:52.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 shutdown_connections 2023-10-18T17:59:52.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc32805a510 0x7fc32805c9d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:52.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3440b8bf0 0x7fc3440685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:52.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3440b9690 0x7fc3440680e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:52.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 --2- 172.21.15.72:0/503798562 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc344065d30 0x7fc3440b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:52.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 >> 172.21.15.72:0/503798562 conn(0x7fc3440620d0 msgr2=0x7fc344148f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 shutdown_connections 2023-10-18T17:59:52.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:52.140+0000 7fc34bfdb700 1 -- 172.21.15.72:0/503798562 wait complete. 2023-10-18T17:59:52.167 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:54.565 INFO:tasks.ceph.mds.d.smithi179.stderr:terminate called after throwing an instance of 'boost::wrapexcept' 2023-10-18T17:59:54.566 INFO:tasks.ceph.mds.d.smithi179.stderr: what(): boost::bad_get: failed value get using boost::get 2023-10-18T17:59:54.566 INFO:tasks.ceph.mds.d.smithi179.stderr:*** Caught signal (Aborted) ** 2023-10-18T17:59:54.566 INFO:tasks.ceph.mds.d.smithi179.stderr: in thread 7fcd40166700 thread_name:ms_dispatch 2023-10-18T17:59:54.567 INFO:tasks.ceph.mds.d.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:54.567 INFO:tasks.ceph.mds.d.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7fcd453c5420] 2023-10-18T17:59:54.567 INFO:tasks.ceph.mds.d.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:54.568 INFO:tasks.ceph.mds.d.smithi179.stderr: 3: abort() 2023-10-18T17:59:54.568 INFO:tasks.ceph.mds.d.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7fcd45267911] 2023-10-18T17:59:54.568 INFO:tasks.ceph.mds.d.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7fcd4527338c] 2023-10-18T17:59:54.568 INFO:tasks.ceph.mds.d.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7fcd452733f7] 2023-10-18T17:59:54.569 INFO:tasks.ceph.mds.d.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7fcd452736a9] 2023-10-18T17:59:54.569 INFO:tasks.ceph.mds.d.smithi179.stderr: 8: ceph-mds(+0x185107) [0x563143867107] 2023-10-18T17:59:54.569 INFO:tasks.ceph.mds.d.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x56314386772d] 2023-10-18T17:59:54.569 INFO:tasks.ceph.mds.d.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5631438d909f] 2023-10-18T17:59:54.569 INFO:tasks.ceph.mds.d.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5631438daa95] 2023-10-18T17:59:54.570 INFO:tasks.ceph.mds.d.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5631438a5b10] 2023-10-18T17:59:54.570 INFO:tasks.ceph.mds.d.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5631438a9abf] 2023-10-18T17:59:54.570 INFO:tasks.ceph.mds.d.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5631438aa456] 2023-10-18T17:59:54.570 INFO:tasks.ceph.mds.d.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7fcd458cb710] 2023-10-18T17:59:54.570 INFO:tasks.ceph.mds.d.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7fcd458c8faf] 2023-10-18T17:59:54.571 INFO:tasks.ceph.mds.d.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd45997b21] 2023-10-18T17:59:54.571 INFO:tasks.ceph.mds.d.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7fcd453b9609] 2023-10-18T17:59:54.571 INFO:tasks.ceph.mds.d.smithi179.stderr: 19: clone() 2023-10-18T17:59:54.572 INFO:tasks.ceph.mds.d.smithi179.stderr:2023-10-18T17:59:54.563+0000 7fcd40166700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:54.572 INFO:tasks.ceph.mds.d.smithi179.stderr: in thread 7fcd40166700 thread_name:ms_dispatch 2023-10-18T17:59:54.572 INFO:tasks.ceph.mds.d.smithi179.stderr: 2023-10-18T17:59:54.572 INFO:tasks.ceph.mds.d.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:54.573 INFO:tasks.ceph.mds.d.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7fcd453c5420] 2023-10-18T17:59:54.573 INFO:tasks.ceph.mds.d.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:54.573 INFO:tasks.ceph.mds.d.smithi179.stderr: 3: abort() 2023-10-18T17:59:54.573 INFO:tasks.ceph.mds.d.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7fcd45267911] 2023-10-18T17:59:54.573 INFO:tasks.ceph.mds.d.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7fcd4527338c] 2023-10-18T17:59:54.574 INFO:tasks.ceph.mds.d.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7fcd452733f7] 2023-10-18T17:59:54.574 INFO:tasks.ceph.mds.d.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7fcd452736a9] 2023-10-18T17:59:54.574 INFO:tasks.ceph.mds.d.smithi179.stderr: 8: ceph-mds(+0x185107) [0x563143867107] 2023-10-18T17:59:54.574 INFO:tasks.ceph.mds.d.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x56314386772d] 2023-10-18T17:59:54.574 INFO:tasks.ceph.mds.d.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5631438d909f] 2023-10-18T17:59:54.575 INFO:tasks.ceph.mds.d.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5631438daa95] 2023-10-18T17:59:54.575 INFO:tasks.ceph.mds.d.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5631438a5b10] 2023-10-18T17:59:54.575 INFO:tasks.ceph.mds.d.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5631438a9abf] 2023-10-18T17:59:54.575 INFO:tasks.ceph.mds.d.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5631438aa456] 2023-10-18T17:59:54.575 INFO:tasks.ceph.mds.d.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7fcd458cb710] 2023-10-18T17:59:54.576 INFO:tasks.ceph.mds.d.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7fcd458c8faf] 2023-10-18T17:59:54.576 INFO:tasks.ceph.mds.d.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd45997b21] 2023-10-18T17:59:54.576 INFO:tasks.ceph.mds.d.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7fcd453b9609] 2023-10-18T17:59:54.576 INFO:tasks.ceph.mds.d.smithi179.stderr: 19: clone() 2023-10-18T17:59:54.576 INFO:tasks.ceph.mds.d.smithi179.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:54.577 INFO:tasks.ceph.mds.d.smithi179.stderr: 2023-10-18T17:59:54.577 INFO:tasks.ceph.mds.d.smithi179.stderr: 0> 2023-10-18T17:59:54.563+0000 7fcd40166700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:54.577 INFO:tasks.ceph.mds.d.smithi179.stderr: in thread 7fcd40166700 thread_name:ms_dispatch 2023-10-18T17:59:54.577 INFO:tasks.ceph.mds.d.smithi179.stderr: 2023-10-18T17:59:54.577 INFO:tasks.ceph.mds.d.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:54.578 INFO:tasks.ceph.mds.d.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7fcd453c5420] 2023-10-18T17:59:54.578 INFO:tasks.ceph.mds.d.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:54.578 INFO:tasks.ceph.mds.d.smithi179.stderr: 3: abort() 2023-10-18T17:59:54.578 INFO:tasks.ceph.mds.d.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7fcd45267911] 2023-10-18T17:59:54.578 INFO:tasks.ceph.mds.d.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7fcd4527338c] 2023-10-18T17:59:54.579 INFO:tasks.ceph.mds.d.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7fcd452733f7] 2023-10-18T17:59:54.579 INFO:tasks.ceph.mds.d.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7fcd452736a9] 2023-10-18T17:59:54.579 INFO:tasks.ceph.mds.d.smithi179.stderr: 8: ceph-mds(+0x185107) [0x563143867107] 2023-10-18T17:59:54.579 INFO:tasks.ceph.mds.d.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x56314386772d] 2023-10-18T17:59:54.579 INFO:tasks.ceph.mds.d.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5631438d909f] 2023-10-18T17:59:54.579 INFO:tasks.ceph.mds.d.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5631438daa95] 2023-10-18T17:59:54.580 INFO:tasks.ceph.mds.d.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5631438a5b10] 2023-10-18T17:59:54.580 INFO:tasks.ceph.mds.d.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5631438a9abf] 2023-10-18T17:59:54.580 INFO:tasks.ceph.mds.d.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5631438aa456] 2023-10-18T17:59:54.580 INFO:tasks.ceph.mds.d.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7fcd458cb710] 2023-10-18T17:59:54.580 INFO:tasks.ceph.mds.d.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7fcd458c8faf] 2023-10-18T17:59:54.581 INFO:tasks.ceph.mds.d.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd45997b21] 2023-10-18T17:59:54.581 INFO:tasks.ceph.mds.d.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7fcd453b9609] 2023-10-18T17:59:54.581 INFO:tasks.ceph.mds.d.smithi179.stderr: 19: clone() 2023-10-18T17:59:54.581 INFO:tasks.ceph.mds.d.smithi179.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:54.581 INFO:tasks.ceph.mds.d.smithi179.stderr: 2023-10-18T17:59:54.582 INFO:tasks.ceph.mds.d.smithi179.stderr: 0> 2023-10-18T17:59:54.563+0000 7fcd40166700 -1 *** Caught signal (Aborted) ** 2023-10-18T17:59:54.582 INFO:tasks.ceph.mds.d.smithi179.stderr: in thread 7fcd40166700 thread_name:ms_dispatch 2023-10-18T17:59:54.582 INFO:tasks.ceph.mds.d.smithi179.stderr: 2023-10-18T17:59:54.582 INFO:tasks.ceph.mds.d.smithi179.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T17:59:54.582 INFO:tasks.ceph.mds.d.smithi179.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7fcd453c5420] 2023-10-18T17:59:54.583 INFO:tasks.ceph.mds.d.smithi179.stderr: 2: gsignal() 2023-10-18T17:59:54.583 INFO:tasks.ceph.mds.d.smithi179.stderr: 3: abort() 2023-10-18T17:59:54.583 INFO:tasks.ceph.mds.d.smithi179.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7fcd45267911] 2023-10-18T17:59:54.583 INFO:tasks.ceph.mds.d.smithi179.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7fcd4527338c] 2023-10-18T17:59:54.583 INFO:tasks.ceph.mds.d.smithi179.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7fcd452733f7] 2023-10-18T17:59:54.584 INFO:tasks.ceph.mds.d.smithi179.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7fcd452736a9] 2023-10-18T17:59:54.584 INFO:tasks.ceph.mds.d.smithi179.stderr: 8: ceph-mds(+0x185107) [0x563143867107] 2023-10-18T17:59:54.584 INFO:tasks.ceph.mds.d.smithi179.stderr: 9: ceph-mds(+0x18572d) [0x56314386772d] 2023-10-18T17:59:54.584 INFO:tasks.ceph.mds.d.smithi179.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x5631438d909f] 2023-10-18T17:59:54.584 INFO:tasks.ceph.mds.d.smithi179.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x5631438daa95] 2023-10-18T17:59:54.585 INFO:tasks.ceph.mds.d.smithi179.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x5631438a5b10] 2023-10-18T17:59:54.585 INFO:tasks.ceph.mds.d.smithi179.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x5631438a9abf] 2023-10-18T17:59:54.585 INFO:tasks.ceph.mds.d.smithi179.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x5631438aa456] 2023-10-18T17:59:54.585 INFO:tasks.ceph.mds.d.smithi179.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7fcd458cb710] 2023-10-18T17:59:54.585 INFO:tasks.ceph.mds.d.smithi179.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7fcd458c8faf] 2023-10-18T17:59:54.585 INFO:tasks.ceph.mds.d.smithi179.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd45997b21] 2023-10-18T17:59:54.586 INFO:tasks.ceph.mds.d.smithi179.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7fcd453b9609] 2023-10-18T17:59:54.586 INFO:tasks.ceph.mds.d.smithi179.stderr: 19: clone() 2023-10-18T17:59:54.586 INFO:tasks.ceph.mds.d.smithi179.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T17:59:54.586 INFO:tasks.ceph.mds.d.smithi179.stderr: 2023-10-18T17:59:54.839 INFO:tasks.ceph.mds.d.smithi179.stderr:daemon-helper: command crashed with signal 6 2023-10-18T17:59:55.168 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd27409b700 1 Processor -- start 2023-10-18T17:59:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd27409b700 1 -- start start 2023-10-18T17:59:55.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd27409b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 0x7fd26c0bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:55.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd27409b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd26c0bde20 0x7fd26c151410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd27409b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c065cd0 con 0x7fd26c067ff0 2023-10-18T17:59:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd27409b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26c066160 con 0x7fd26c0bde20 2023-10-18T17:59:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd27409b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c154750 con 0x7fd26c0666b0 2023-10-18T17:59:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd271e37700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fd26c067ff0 0x7fd26c0683d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:53896/0 (socket says 172.21.15.72:53896) 2023-10-18T17:59:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd271e37700 1 -- 172.21.15.72:0/3188714633 learned_addr learned my addr 172.21.15.72:0/3188714633 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T17:59:55.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd271636700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 0x7fd26c0bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:55.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd272638700 1 --2- 172.21.15.72:0/3188714633 >> v2:172.21.15.179:3300/0 conn(0x7fd26c0bde20 0x7fd26c151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:55.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299999905 0 0) 0x7fd26c065cd0 con 0x7fd26c067ff0 2023-10-18T17:59:55.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.328+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2580033f0 con 0x7fd26c067ff0 2023-10-18T17:59:55.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 <== mon.1 v1:172.21.15.179:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4174187514 0 0) 0x7fd2580033f0 con 0x7fd26c067ff0 2023-10-18T17:59:55.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 msgr2=0x7fd26c0bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:55.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 0x7fd26c0bd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 >> v2:172.21.15.179:3300/0 conn(0x7fd26c0bde20 msgr2=0x7fd26c151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:55.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 --2- 172.21.15.72:0/3188714633 >> v2:172.21.15.179:3300/0 conn(0x7fd26c0bde20 0x7fd26c151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 --> v1:172.21.15.179:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c065100 con 0x7fd26c067ff0 2023-10-18T17:59:55.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 <== mon.1 v1:172.21.15.179:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1723103133 0 0) 0x7fd268001470 con 0x7fd26c067ff0 2023-10-18T17:59:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271636700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 0x7fd26c0bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:59:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd272638700 1 --2- 172.21.15.72:0/3188714633 >> v2:172.21.15.179:3300/0 conn(0x7fd26c0bde20 0x7fd26c151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:59:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 >> v1:172.21.15.179:6789/0 conn(0x7fd26c067ff0 legacy=0x7fd26c0683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2580038e0 0x7fd258003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:55.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 0x7fd258004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:55.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd258004b80 0x7fd258005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:55.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c066160 con 0x7fd2580038e0 2023-10-18T17:59:55.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c154750 con 0x7fd258004b80 2023-10-18T17:59:55.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25800b8e0 con 0x7fd26c0666b0 2023-10-18T17:59:55.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271e37700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2580038e0 0x7fd258003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:55.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271636700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 0x7fd258004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:55.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd272638700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd258004b80 0x7fd258005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:55.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271e37700 1 -- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 msgr2=0x7fd258004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:55.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271e37700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 0x7fd258004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271e37700 1 -- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd258004b80 msgr2=0x7fd258005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T17:59:55.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271e37700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd258004b80 0x7fd258005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271e37700 1 -- 172.21.15.72:0/3188714633 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0092a0 con 0x7fd2580038e0 2023-10-18T17:59:55.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271636700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd26c0666b0 0x7fd258004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T17:59:55.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd271e37700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2580038e0 0x7fd258003d00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fd26800c210 tx=0x7fd268001e90).ready entity=mon.0 client_cookie=b1d1fa0b59a8ed6a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:55.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd268007a60 con 0x7fd2580038e0 2023-10-18T17:59:55.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26800cae0 con 0x7fd2580038e0 2023-10-18T17:59:55.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd270e35700 1 -- 172.21.15.72:0/3188714633 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800ce80 con 0x7fd2580038e0 2023-10-18T17:59:55.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd27409b700 1 -- 172.21.15.72:0/3188714633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2580038e0 msgr2=0x7fd258003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:55.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd27409b700 1 --2- 172.21.15.72:0/3188714633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd2580038e0 0x7fd258003d00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fd26800c210 tx=0x7fd268001e90).stop 2023-10-18T17:59:55.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd272638700 1 -- 172.21.15.72:0/3188714633 reap_dead start 2023-10-18T17:59:55.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd27409b700 1 -- 172.21.15.72:0/3188714633 shutdown_connections 2023-10-18T17:59:55.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd27409b700 1 -- 172.21.15.72:0/3188714633 >> 172.21.15.72:0/3188714633 conn(0x7fd26c0620d0 msgr2=0x7fd26c0624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:55.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd27409b700 1 -- 172.21.15.72:0/3188714633 shutdown_connections 2023-10-18T17:59:55.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.332+0000 7fd27409b700 1 -- 172.21.15.72:0/3188714633 wait complete. 2023-10-18T17:59:55.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd27409b700 1 Processor -- start 2023-10-18T17:59:55.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd27409b700 1 -- start start 2023-10-18T17:59:55.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd27409b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2580038e0 0x7fd26c1683f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:55.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd27409b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd26c159620 0x7fd26c164d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:55.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd27409b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd26c156150 0x7fd26c0baac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:55.349 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd27409b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0b9c50 con 0x7fd26c156150 2023-10-18T17:59:55.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd27409b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0bafb0 con 0x7fd26c159620 2023-10-18T17:59:55.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd27409b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0c0270 con 0x7fd2580038e0 2023-10-18T17:59:55.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd272638700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd26c156150 0x7fd26c0baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:55.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd271636700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd26c159620 0x7fd26c164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:55.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd271636700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd26c159620 0x7fd26c164d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:36720/0 (socket says 172.21.15.72:36720) 2023-10-18T17:59:55.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd271636700 1 -- 172.21.15.72:0/2886912935 learned_addr learned my addr 172.21.15.72:0/2886912935 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:55.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd271e37700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2580038e0 0x7fd26c1683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:55.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.336+0000 7fd272638700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd26c156150 0x7fd26c0baac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35088/0 (socket says 172.21.15.72:35088) 2023-10-18T17:59:55.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd272638700 1 -- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2580038e0 msgr2=0x7fd26c1683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:55.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd272638700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2580038e0 0x7fd26c1683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd272638700 1 -- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd26c159620 msgr2=0x7fd26c164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:55.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd272638700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd26c159620 0x7fd26c164d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd272638700 1 -- 172.21.15.72:0/2886912935 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c0c04c0 con 0x7fd26c156150 2023-10-18T17:59:55.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd272638700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd26c156150 0x7fd26c0baac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fd26c0be200 tx=0x7fd26400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:55.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd261ffb700 1 -- 172.21.15.72:0/2886912935 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400e040 con 0x7fd26c156150 2023-10-18T17:59:55.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd261ffb700 1 -- 172.21.15.72:0/2886912935 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26400f040 con 0x7fd26c156150 2023-10-18T17:59:55.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd261ffb700 1 -- 172.21.15.72:0/2886912935 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400fd70 con 0x7fd26c156150 2023-10-18T17:59:55.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c167910 con 0x7fd26c156150 2023-10-18T17:59:55.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26c167ed0 con 0x7fd26c156150 2023-10-18T17:59:55.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26c1680c0 con 0x7fd26c156150 2023-10-18T17:59:55.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.340+0000 7fd261ffb700 1 -- 172.21.15.72:0/2886912935 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd264015660 con 0x7fd26c156150 2023-10-18T17:59:55.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.344+0000 7fd261ffb700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd25005a510 0x7fd25005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:55.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.344+0000 7fd261ffb700 1 -- 172.21.15.72:0/2886912935 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6494+0+0 (secure 0 0 0) 0x7fd2640753b0 con 0x7fd26c156150 2023-10-18T17:59:55.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.344+0000 7fd271e37700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd25005a510 0x7fd25005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:55.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.344+0000 7fd271e37700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd25005a510 0x7fd25005c9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd268001ed0 tx=0x7fd268015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:55.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.348+0000 7fd261ffb700 1 -- 172.21.15.72:0/2886912935 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fd2640489c0 con 0x7fd26c156150 2023-10-18T17:59:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.544+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd26c06c190 con 0x7fd26c156150 2023-10-18T17:59:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.544+0000 7fd261ffb700 1 -- 172.21.15.72:0/2886912935 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fd26404bfe0 con 0x7fd26c156150 2023-10-18T17:59:55.552 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:55.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.548+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd25005a510 msgr2=0x7fd25005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.548+0000 7fd27409b700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd25005a510 0x7fd25005c9d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fd268001ed0 tx=0x7fd268015000).stop 2023-10-18T17:59:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.548+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd26c156150 msgr2=0x7fd26c0baac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.548+0000 7fd27409b700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd26c156150 0x7fd26c0baac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fd26c0be200 tx=0x7fd26400b040).stop 2023-10-18T17:59:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.552+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 shutdown_connections 2023-10-18T17:59:55.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.552+0000 7fd27409b700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd25005a510 0x7fd25005c9d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.552+0000 7fd27409b700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd26c156150 0x7fd26c0baac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.552+0000 7fd27409b700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd26c159620 0x7fd26c164d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.552+0000 7fd27409b700 1 --2- 172.21.15.72:0/2886912935 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd2580038e0 0x7fd26c1683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:55.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.552+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 >> 172.21.15.72:0/2886912935 conn(0x7fd26c0620d0 msgr2=0x7fd26c154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:55.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.552+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 shutdown_connections 2023-10-18T17:59:55.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:55.552+0000 7fd27409b700 1 -- 172.21.15.72:0/2886912935 wait complete. 2023-10-18T17:59:55.577 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T17:59:56.545 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:59:56.553 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T17:59:58.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~87s 2023-10-18T17:59:58.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~33s 2023-10-18T17:59:58.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-10-18T17:59:58.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-18T17:59:58.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~46s 2023-10-18T17:59:58.579 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T17:59:58.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd942bb9700 1 Processor -- start 2023-10-18T17:59:58.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd942bb9700 1 -- start start 2023-10-18T17:59:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd942bb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1517c0 0x7fd93c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd942bb9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93c154b00 con 0x7fd93c1513e0 2023-10-18T17:59:58.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd942bb9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93c154f90 con 0x7fd93c0630e0 2023-10-18T17:59:58.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd942bb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c069380 con 0x7fd93c1517c0 2023-10-18T17:59:58.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd9413b6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1517c0 0x7fd93c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:58.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd9413b6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1517c0 0x7fd93c06a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:60996/0 (socket says 172.21.15.72:60996) 2023-10-18T17:59:58.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd9413b6700 1 -- 172.21.15.72:0/238253286 learned_addr learned my addr 172.21.15.72:0/238253286 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:58.749 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd941bb7700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd93c1513e0 0x7fd93c067d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42326/0 (socket says 172.21.15.72:42326) 2023-10-18T17:59:58.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.740+0000 7fd9423b8700 1 --1- 172.21.15.72:0/238253286 >> v1:172.21.15.179:6789/0 conn(0x7fd93c0630e0 0x7fd93c065700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:53904/0 (socket says 172.21.15.72:53904) 2023-10-18T17:59:58.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd9413b6700 1 -- 172.21.15.72:0/238253286 >> v1:172.21.15.179:6789/0 conn(0x7fd93c0630e0 legacy=0x7fd93c065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:58.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd9413b6700 1 -- 172.21.15.72:0/238253286 >> v1:172.21.15.72:6789/0 conn(0x7fd93c1513e0 legacy=0x7fd93c067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:58.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd940bb5700 1 -- 172.21.15.72:0/238253286 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764945497 0 0) 0x7fd93c154f90 con 0x7fd93c0630e0 2023-10-18T17:59:58.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd9413b6700 1 -- 172.21.15.72:0/238253286 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c068570 con 0x7fd93c1517c0 2023-10-18T17:59:58.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd9413b6700 1 --2- 172.21.15.72:0/238253286 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1517c0 0x7fd93c06a040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fd92c009830 tx=0x7fd92c0090f0).ready entity=mon.2 client_cookie=eb7879dbb7324f9e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd940bb5700 1 -- 172.21.15.72:0/238253286 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00b640 con 0x7fd93c1517c0 2023-10-18T17:59:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd940bb5700 1 -- 172.21.15.72:0/238253286 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c00b7a0 con 0x7fd93c1517c0 2023-10-18T17:59:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd940bb5700 1 -- 172.21.15.72:0/238253286 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00bb20 con 0x7fd93c1517c0 2023-10-18T17:59:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd942bb9700 1 -- 172.21.15.72:0/238253286 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1517c0 msgr2=0x7fd93c06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:58.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd942bb9700 1 --2- 172.21.15.72:0/238253286 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1517c0 0x7fd93c06a040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fd92c009830 tx=0x7fd92c0090f0).stop 2023-10-18T17:59:58.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd942bb9700 1 -- 172.21.15.72:0/238253286 shutdown_connections 2023-10-18T17:59:58.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd942bb9700 1 --2- 172.21.15.72:0/238253286 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1517c0 0x7fd93c06a040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:58.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd942bb9700 1 -- 172.21.15.72:0/238253286 >> 172.21.15.72:0/238253286 conn(0x7fd93c062ba0 msgr2=0x7fd93c0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:58.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd942bb9700 1 -- 172.21.15.72:0/238253286 shutdown_connections 2023-10-18T17:59:58.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.744+0000 7fd942bb9700 1 -- 172.21.15.72:0/238253286 wait complete. 2023-10-18T17:59:58.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd942bb9700 1 Processor -- start 2023-10-18T17:59:58.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd942bb9700 1 -- start start 2023-10-18T17:59:58.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd942bb9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd93c0630e0 0x7fd93c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:58.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd942bb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1513e0 0x7fd93c156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:58.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd942bb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd93c1517c0 0x7fd93c0bba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:58.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd942bb9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c068b30 con 0x7fd93c0630e0 2023-10-18T17:59:58.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd942bb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c157720 con 0x7fd93c1517c0 2023-10-18T17:59:58.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd942bb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c157a00 con 0x7fd93c1513e0 2023-10-18T17:59:58.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd941bb7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd93c0630e0 0x7fd93c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:58.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd941bb7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd93c0630e0 0x7fd93c155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35090/0 (socket says 172.21.15.72:35090) 2023-10-18T17:59:58.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd941bb7700 1 -- 172.21.15.72:0/903757238 learned_addr learned my addr 172.21.15.72:0/903757238 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T17:59:58.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd9423b8700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd93c1517c0 0x7fd93c0bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:58.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd9413b6700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1513e0 0x7fd93c156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:58.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd941bb7700 1 -- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1513e0 msgr2=0x7fd93c156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:58.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd941bb7700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1513e0 0x7fd93c156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:58.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd941bb7700 1 -- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd93c1517c0 msgr2=0x7fd93c0bba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:58.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd941bb7700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd93c1517c0 0x7fd93c0bba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:58.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd941bb7700 1 -- 172.21.15.72:0/903757238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c009370 con 0x7fd93c0630e0 2023-10-18T17:59:58.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.748+0000 7fd941bb7700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd93c0630e0 0x7fd93c155e90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fd938001eb0 tx=0x7fd938007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:58.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.752+0000 7fd931ffb700 1 -- 172.21.15.72:0/903757238 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9380072e0 con 0x7fd93c0630e0 2023-10-18T17:59:58.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.752+0000 7fd931ffb700 1 -- 172.21.15.72:0/903757238 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd938007440 con 0x7fd93c0630e0 2023-10-18T17:59:58.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.752+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93c15bdd0 con 0x7fd93c0630e0 2023-10-18T17:59:58.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.752+0000 7fd931ffb700 1 -- 172.21.15.72:0/903757238 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9380058f0 con 0x7fd93c0630e0 2023-10-18T17:59:58.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.752+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93c157ce0 con 0x7fd93c0630e0 2023-10-18T17:59:58.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.752+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93c158aa0 con 0x7fd93c0630e0 2023-10-18T17:59:58.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.752+0000 7fd931ffb700 1 -- 172.21.15.72:0/903757238 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd938005ce0 con 0x7fd93c0630e0 2023-10-18T17:59:58.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.756+0000 7fd931ffb700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd92005a510 0x7fd92005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T17:59:58.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.756+0000 7fd931ffb700 1 -- 172.21.15.72:0/903757238 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6494+0+0 (secure 0 0 0) 0x7fd938074110 con 0x7fd93c0630e0 2023-10-18T17:59:58.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.756+0000 7fd9413b6700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd92005a510 0x7fd92005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T17:59:58.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.756+0000 7fd9413b6700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd92005a510 0x7fd92005c9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd92c008000 tx=0x7fd92c017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T17:59:58.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.760+0000 7fd931ffb700 1 -- 172.21.15.72:0/903757238 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fd938047780 con 0x7fd93c0630e0 2023-10-18T17:59:58.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.956+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd93c046f00 con 0x7fd93c0630e0 2023-10-18T17:59:58.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.956+0000 7fd931ffb700 1 -- 172.21.15.72:0/903757238 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fd938018080 con 0x7fd93c0630e0 2023-10-18T17:59:58.962 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T17:59:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd92005a510 msgr2=0x7fd92005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd92005a510 0x7fd92005c9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd92c008000 tx=0x7fd92c017000).stop 2023-10-18T17:59:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd93c0630e0 msgr2=0x7fd93c155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T17:59:58.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd93c0630e0 0x7fd93c155e90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fd938001eb0 tx=0x7fd938007df0).stop 2023-10-18T17:59:58.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 shutdown_connections 2023-10-18T17:59:58.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd92005a510 0x7fd92005c9d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:58.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd93c1517c0 0x7fd93c0bba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd93c1513e0 0x7fd93c156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 --2- 172.21.15.72:0/903757238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd93c0630e0 0x7fd93c155e90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T17:59:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 >> 172.21.15.72:0/903757238 conn(0x7fd93c062ba0 msgr2=0x7fd93c0bd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T17:59:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 shutdown_connections 2023-10-18T17:59:58.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T17:59:58.960+0000 7fd942bb9700 1 -- 172.21.15.72:0/903757238 wait complete. 2023-10-18T17:59:58.986 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:01.988 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:02.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.144+0000 7f30fecb9700 1 Processor -- start 2023-10-18T18:00:02.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.144+0000 7f30fecb9700 1 -- start start 2023-10-18T18:00:02.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fecb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30f806a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:02.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fecb9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f8068910 con 0x7f30f8068150 2023-10-18T18:00:02.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fecb9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f8150780 con 0x7f30f8067d30 2023-10-18T18:00:02.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fecb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8150c10 con 0x7f30f814d170 2023-10-18T18:00:02.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fe4b8700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f30f8068150 0x7f30f8065700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51434/0 (socket says 172.21.15.72:51434) 2023-10-18T18:00:02.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fe4b8700 1 -- 172.21.15.72:0/1392475455 learned_addr learned my addr 172.21.15.72:0/1392475455 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:02.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fdcb7700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30f806a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:02.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634359809 0 0) 0x7f30f8068910 con 0x7f30f8068150 2023-10-18T18:00:02.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e40033f0 con 0x7f30f8068150 2023-10-18T18:00:02.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760296460 0 0) 0x7f30f8150780 con 0x7f30f8067d30 2023-10-18T18:00:02.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f8068910 con 0x7f30f8067d30 2023-10-18T18:00:02.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2887137468 0 0) 0x7f30e40033f0 con 0x7f30f8068150 2023-10-18T18:00:02.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 msgr2=0x7f30f806a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:02.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30f806a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:02.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 >> v1:172.21.15.179:6789/0 conn(0x7f30f8067d30 legacy=0x7f30f80630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:02.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8068630 con 0x7f30f8068150 2023-10-18T18:00:02.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1723103133 0 0) 0x7f30f00017f0 con 0x7f30f8068150 2023-10-18T18:00:02.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 >> v1:172.21.15.72:6789/0 conn(0x7f30f8068150 legacy=0x7f30f8065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:02.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30e4003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:02.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30e4003fb0 0x7f30e4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:02.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30e4004b80 0x7f30e4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:02.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8150c10 con 0x7f30e4004b80 2023-10-18T18:00:02.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e400b990 con 0x7f30e4003fb0 2023-10-18T18:00:02.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e400bb50 con 0x7f30f814d170 2023-10-18T18:00:02.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fe4b8700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30e4004b80 0x7f30e4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:02.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fdcb7700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30e4003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:02.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.148+0000 7f30fd4b6700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30e4003fb0 0x7f30e4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:02.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fdcb7700 1 -- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30e4003fb0 msgr2=0x7f30e4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:02.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fdcb7700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30e4003fb0 0x7f30e4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:02.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fdcb7700 1 -- 172.21.15.72:0/1392475455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30e4004b80 msgr2=0x7f30e4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:02.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fdcb7700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30e4004b80 0x7f30e4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fdcb7700 1 -- 172.21.15.72:0/1392475455 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e40033f0 con 0x7f30f814d170 2023-10-18T18:00:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fe4b8700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30e4004b80 0x7f30e4005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:00:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fdcb7700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30e4003aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f30f4009a80 tx=0x7f30f4009060).ready entity=mon.2 client_cookie=bf40ab816077a94d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4009430 con 0x7f30f814d170 2023-10-18T18:00:02.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f400e640 con 0x7f30f814d170 2023-10-18T18:00:02.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fccb5700 1 -- 172.21.15.72:0/1392475455 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f400e9c0 con 0x7f30f814d170 2023-10-18T18:00:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fecb9700 1 -- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 msgr2=0x7f30e4003aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fecb9700 1 --2- 172.21.15.72:0/1392475455 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30e4003aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f30f4009a80 tx=0x7f30f4009060).stop 2023-10-18T18:00:02.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fe4b8700 1 -- 172.21.15.72:0/1392475455 reap_dead start 2023-10-18T18:00:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fecb9700 1 -- 172.21.15.72:0/1392475455 shutdown_connections 2023-10-18T18:00:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fecb9700 1 -- 172.21.15.72:0/1392475455 >> 172.21.15.72:0/1392475455 conn(0x7f30f8062ba0 msgr2=0x7f30f80c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fecb9700 1 -- 172.21.15.72:0/1392475455 shutdown_connections 2023-10-18T18:00:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fecb9700 1 -- 172.21.15.72:0/1392475455 wait complete. 2023-10-18T18:00:02.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fecb9700 1 Processor -- start 2023-10-18T18:00:02.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.152+0000 7f30fecb9700 1 -- start start 2023-10-18T18:00:02.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fecb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30f80631c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:02.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fecb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8157d50 0x7f30f8151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:02.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fecb9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30f80c0540 0x7f30f80bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:02.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fecb9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8157920 con 0x7f30f80c0540 2023-10-18T18:00:02.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fecb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8065ff0 con 0x7f30f8157d50 2023-10-18T18:00:02.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fecb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8151480 con 0x7f30f814d170 2023-10-18T18:00:02.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fe4b8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30f80c0540 0x7f30f80bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:02.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fdcb7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30f80631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:02.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fdcb7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30f80631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:43808/0 (socket says 172.21.15.72:43808) 2023-10-18T18:00:02.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fdcb7700 1 -- 172.21.15.72:0/3343920939 learned_addr learned my addr 172.21.15.72:0/3343920939 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:02.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fe4b8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30f80c0540 0x7f30f80bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35322/0 (socket says 172.21.15.72:35322) 2023-10-18T18:00:02.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fd4b6700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8157d50 0x7f30f8151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:02.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fe4b8700 1 -- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 msgr2=0x7f30f80631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:02.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fe4b8700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30f80631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:02.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fe4b8700 1 -- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8157d50 msgr2=0x7f30f8151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:02.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fe4b8700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8157d50 0x7f30f8151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:02.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fe4b8700 1 -- 172.21.15.72:0/3343920939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8151150 con 0x7f30f80c0540 2023-10-18T18:00:02.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fdcb7700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30f80631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:00:02.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fe4b8700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30f80c0540 0x7f30f80bfb70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f30f0007fd0 tx=0x7f30f0007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:02.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30edffb700 1 -- 172.21.15.72:0/3343920939 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000c960 con 0x7f30f80c0540 2023-10-18T18:00:02.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30edffb700 1 -- 172.21.15.72:0/3343920939 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f000cac0 con 0x7f30f80c0540 2023-10-18T18:00:02.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f8151960 con 0x7f30f80c0540 2023-10-18T18:00:02.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30edffb700 1 -- 172.21.15.72:0/3343920939 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000ce80 con 0x7f30f80c0540 2023-10-18T18:00:02.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.156+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f81608a0 con 0x7f30f80c0540 2023-10-18T18:00:02.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.160+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f8046f00 con 0x7f30f80c0540 2023-10-18T18:00:02.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.160+0000 7f30edffb700 1 -- 172.21.15.72:0/3343920939 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f30f0015620 con 0x7f30f80c0540 2023-10-18T18:00:02.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.160+0000 7f30edffb700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f30dc05a510 0x7f30dc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:02.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.160+0000 7f30edffb700 1 -- 172.21.15.72:0/3343920939 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6494+0+0 (secure 0 0 0) 0x7f30f00747a0 con 0x7f30f80c0540 2023-10-18T18:00:02.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.160+0000 7f30fdcb7700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f30dc05a510 0x7f30dc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:02.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.164+0000 7f30fdcb7700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f30dc05a510 0x7f30dc05c9d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f30f4006d50 tx=0x7f30f401a000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:02.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.168+0000 7f30edffb700 1 -- 172.21.15.72:0/3343920939 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f30f0047db0 con 0x7f30f80c0540 2023-10-18T18:00:02.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.360+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f30f80ac460 con 0x7f30f80c0540 2023-10-18T18:00:02.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.360+0000 7f30edffb700 1 -- 172.21.15.72:0/3343920939 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f30f001d030 con 0x7f30f80c0540 2023-10-18T18:00:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:02.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f30dc05a510 msgr2=0x7f30dc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:02.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f30dc05a510 0x7f30dc05c9d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f30f4006d50 tx=0x7f30f401a000).stop 2023-10-18T18:00:02.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30f80c0540 msgr2=0x7f30f80bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:02.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30f80c0540 0x7f30f80bfb70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f30f0007fd0 tx=0x7f30f0007f40).stop 2023-10-18T18:00:02.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 shutdown_connections 2023-10-18T18:00:02.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f30dc05a510 0x7f30dc05c9d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:02.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30f80c0540 0x7f30f80bfb70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:02.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30f8157d50 0x7f30f8151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:02.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 --2- 172.21.15.72:0/3343920939 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f30f814d170 0x7f30f80631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:02.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 >> 172.21.15.72:0/3343920939 conn(0x7f30f8062ba0 msgr2=0x7f30f8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:02.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 shutdown_connections 2023-10-18T18:00:02.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:02.368+0000 7f30fecb9700 1 -- 172.21.15.72:0/3343920939 wait complete. 2023-10-18T18:00:02.392 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:04.822 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~94s 2023-10-18T18:00:04.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~39s 2023-10-18T18:00:04.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~19s 2023-10-18T18:00:04.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~6s 2023-10-18T18:00:04.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~53s 2023-10-18T18:00:05.393 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:05.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.540+0000 7fa2228e4700 1 Processor -- start 2023-10-18T18:00:05.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2228e4700 1 -- start start 2023-10-18T18:00:05.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2228e4700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa21c1513e0 0x7fa21c067d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:05.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2228e4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1517c0 0x7fa21c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:05.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2228e4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21c154b00 con 0x7fa21c0630e0 2023-10-18T18:00:05.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2228e4700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c069380 con 0x7fa21c1513e0 2023-10-18T18:00:05.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2228e4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c154f90 con 0x7fa21c1517c0 2023-10-18T18:00:05.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2218e2700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa21c1513e0 0x7fa21c067d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:05.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2218e2700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa21c1513e0 0x7fa21c067d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35326/0 (socket says 172.21.15.72:35326) 2023-10-18T18:00:05.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2218e2700 1 -- 172.21.15.72:0/2006145607 learned_addr learned my addr 172.21.15.72:0/2006145607 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:05.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.544+0000 7fa2210e1700 1 --2- 172.21.15.72:0/2006145607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1517c0 0x7fa21c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:05.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2218e2700 1 -- 172.21.15.72:0/2006145607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1517c0 msgr2=0x7fa21c06a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:05.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2208e0700 1 -- 172.21.15.72:0/2006145607 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559381605 0 0) 0x7fa21c154b00 con 0x7fa21c0630e0 2023-10-18T18:00:05.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2218e2700 1 --2- 172.21.15.72:0/2006145607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1517c0 0x7fa21c06a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:05.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2218e2700 1 -- 172.21.15.72:0/2006145607 >> v1:172.21.15.179:6789/0 conn(0x7fa21c0630e0 legacy=0x7fa21c065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:05.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2218e2700 1 -- 172.21.15.72:0/2006145607 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c068570 con 0x7fa21c1513e0 2023-10-18T18:00:05.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2218e2700 1 --2- 172.21.15.72:0/2006145607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c1513e0 0x7fa21c067d30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fa218006fd0 tx=0x7fa2180090f0).ready entity=mon.0 client_cookie=d9c8513a757bda4a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:05.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2208e0700 1 -- 172.21.15.72:0/2006145607 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21800b610 con 0x7fa21c1513e0 2023-10-18T18:00:05.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2208e0700 1 -- 172.21.15.72:0/2006145607 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21800b770 con 0x7fa21c1513e0 2023-10-18T18:00:05.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2208e0700 1 -- 172.21.15.72:0/2006145607 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21800baf0 con 0x7fa21c1513e0 2023-10-18T18:00:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2228e4700 1 -- 172.21.15.72:0/2006145607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c1513e0 msgr2=0x7fa21c067d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2228e4700 1 --2- 172.21.15.72:0/2006145607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c1513e0 0x7fa21c067d30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fa218006fd0 tx=0x7fa2180090f0).stop 2023-10-18T18:00:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2228e4700 1 -- 172.21.15.72:0/2006145607 shutdown_connections 2023-10-18T18:00:05.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2228e4700 1 --2- 172.21.15.72:0/2006145607 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1517c0 0x7fa21c06a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:05.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2228e4700 1 --2- 172.21.15.72:0/2006145607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c1513e0 0x7fa21c067d30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:05.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2228e4700 1 -- 172.21.15.72:0/2006145607 >> 172.21.15.72:0/2006145607 conn(0x7fa21c062ba0 msgr2=0x7fa21c0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:05.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2228e4700 1 -- 172.21.15.72:0/2006145607 shutdown_connections 2023-10-18T18:00:05.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.548+0000 7fa2228e4700 1 -- 172.21.15.72:0/2006145607 wait complete. 2023-10-18T18:00:05.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 Processor -- start 2023-10-18T18:00:05.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 -- start start 2023-10-18T18:00:05.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c0630e0 0x7fa21c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:05.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1513e0 0x7fa21c0bd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:05.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c1517c0 0x7fa21c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:05.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c1555f0 con 0x7fa21c0630e0 2023-10-18T18:00:05.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c157cd0 con 0x7fa21c1517c0 2023-10-18T18:00:05.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c167e50 con 0x7fa21c1513e0 2023-10-18T18:00:05.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2218e2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c0630e0 0x7fa21c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:05.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2218e2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c0630e0 0x7fa21c15bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35336/0 (socket says 172.21.15.72:35336) 2023-10-18T18:00:05.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2218e2700 1 -- 172.21.15.72:0/3083200187 learned_addr learned my addr 172.21.15.72:0/3083200187 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:05.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2210e1700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1513e0 0x7fa21c0bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:05.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2220e3700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c1517c0 0x7fa21c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:05.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2218e2700 1 -- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1513e0 msgr2=0x7fa21c0bd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:05.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2218e2700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1513e0 0x7fa21c0bd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:05.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2218e2700 1 -- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c1517c0 msgr2=0x7fa21c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:05.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2218e2700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c1517c0 0x7fa21c155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:05.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2218e2700 1 -- 172.21.15.72:0/3083200187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa218009370 con 0x7fa21c0630e0 2023-10-18T18:00:05.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2218e2700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c0630e0 0x7fa21c15bcb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fa218009130 tx=0x7fa218017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:05.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa211ffb700 1 -- 172.21.15.72:0/3083200187 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21800beb0 con 0x7fa21c0630e0 2023-10-18T18:00:05.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa211ffb700 1 -- 172.21.15.72:0/3083200187 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21800b8e0 con 0x7fa21c0630e0 2023-10-18T18:00:05.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c168110 con 0x7fa21c0630e0 2023-10-18T18:00:05.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa211ffb700 1 -- 172.21.15.72:0/3083200187 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21801c670 con 0x7fa21c0630e0 2023-10-18T18:00:05.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.552+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c1686d0 con 0x7fa21c0630e0 2023-10-18T18:00:05.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.556+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c046f00 con 0x7fa21c0630e0 2023-10-18T18:00:05.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.556+0000 7fa211ffb700 1 -- 172.21.15.72:0/3083200187 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa21801ca60 con 0x7fa21c0630e0 2023-10-18T18:00:05.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.560+0000 7fa211ffb700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa20005a540 0x7fa20005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:05.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.560+0000 7fa211ffb700 1 -- 172.21.15.72:0/3083200187 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6494+0+0 (secure 0 0 0) 0x7fa2180764a0 con 0x7fa21c0630e0 2023-10-18T18:00:05.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.560+0000 7fa2210e1700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa20005a540 0x7fa20005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:05.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.560+0000 7fa2210e1700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa20005a540 0x7fa20005ca00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa21c0c0dc0 tx=0x7fa20c009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:05.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.564+0000 7fa211ffb700 1 -- 172.21.15.72:0/3083200187 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fa218049b10 con 0x7fa21c0630e0 2023-10-18T18:00:05.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.760+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa21c0a78e0 con 0x7fa21c0630e0 2023-10-18T18:00:05.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.764+0000 7fa211ffb700 1 -- 172.21.15.72:0/3083200187 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fa21804d130 con 0x7fa21c0630e0 2023-10-18T18:00:05.769 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:05.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa20005a540 msgr2=0x7fa20005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:05.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa20005a540 0x7fa20005ca00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa21c0c0dc0 tx=0x7fa20c009060).stop 2023-10-18T18:00:05.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c0630e0 msgr2=0x7fa21c15bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:05.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c0630e0 0x7fa21c15bcb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fa218009130 tx=0x7fa218017000).stop 2023-10-18T18:00:05.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 shutdown_connections 2023-10-18T18:00:05.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa20005a540 0x7fa20005ca00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:05.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c1517c0 0x7fa21c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:05.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa21c1513e0 0x7fa21c0bd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:05.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 --2- 172.21.15.72:0/3083200187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa21c0630e0 0x7fa21c15bcb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:05.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 >> 172.21.15.72:0/3083200187 conn(0x7fa21c062ba0 msgr2=0x7fa21c0656e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:05.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 shutdown_connections 2023-10-18T18:00:05.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:05.768+0000 7fa2228e4700 1 -- 172.21.15.72:0/3083200187 wait complete. 2023-10-18T18:00:05.794 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:08.796 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:08.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.956+0000 7f2034d79700 1 Processor -- start 2023-10-18T18:00:08.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.956+0000 7f2034d79700 1 -- start start 2023-10-18T18:00:08.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f2034d79700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20300630e0 0x7f2030065700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:08.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f2034d79700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2030154b00 con 0x7f20301513e0 2023-10-18T18:00:08.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f2034d79700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203006a750 con 0x7f20301517c0 2023-10-18T18:00:08.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f2034d79700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2030069380 con 0x7f20300630e0 2023-10-18T18:00:08.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202f7fe700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f20301513e0 0x7f2030067d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51450/0 (socket says 172.21.15.72:51450) 2023-10-18T18:00:08.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202f7fe700 1 -- 172.21.15.72:0/1471909210 learned_addr learned my addr 172.21.15.72:0/1471909210 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:08.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202ffff700 1 --2- 172.21.15.72:0/1471909210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20300630e0 0x7f2030065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:08.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202effd700 1 -- 172.21.15.72:0/1471909210 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323891229 0 0) 0x7f203006a750 con 0x7f20301517c0 2023-10-18T18:00:08.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202effd700 1 -- 172.21.15.72:0/1471909210 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f201c0033f0 con 0x7f20301517c0 2023-10-18T18:00:08.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202effd700 1 -- 172.21.15.72:0/1471909210 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934290478 0 0) 0x7f2030154b00 con 0x7f20301513e0 2023-10-18T18:00:08.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202effd700 1 -- 172.21.15.72:0/1471909210 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203006a750 con 0x7f20301513e0 2023-10-18T18:00:08.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202ffff700 1 -- 172.21.15.72:0/1471909210 >> v1:172.21.15.179:6789/0 conn(0x7f20301517c0 legacy=0x7f203006a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:08.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202ffff700 1 -- 172.21.15.72:0/1471909210 >> v1:172.21.15.72:6789/0 conn(0x7f20301513e0 legacy=0x7f2030067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:08.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202ffff700 1 -- 172.21.15.72:0/1471909210 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2030154f90 con 0x7f20300630e0 2023-10-18T18:00:08.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202ffff700 1 --2- 172.21.15.72:0/1471909210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20300630e0 0x7f2030065700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2028009e90 tx=0x7f2028009780).ready entity=mon.2 client_cookie=e2e63658e45eb9f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:08.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202effd700 1 -- 172.21.15.72:0/1471909210 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800e070 con 0x7f20300630e0 2023-10-18T18:00:08.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202effd700 1 -- 172.21.15.72:0/1471909210 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202800bab0 con 0x7f20300630e0 2023-10-18T18:00:08.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.960+0000 7f202effd700 1 -- 172.21.15.72:0/1471909210 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800be30 con 0x7f20300630e0 2023-10-18T18:00:08.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.964+0000 7f2034d79700 1 -- 172.21.15.72:0/1471909210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20300630e0 msgr2=0x7f2030065700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:08.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.964+0000 7f2034d79700 1 --2- 172.21.15.72:0/1471909210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20300630e0 0x7f2030065700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2028009e90 tx=0x7f2028009780).stop 2023-10-18T18:00:08.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.964+0000 7f2034d79700 1 -- 172.21.15.72:0/1471909210 shutdown_connections 2023-10-18T18:00:08.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.964+0000 7f2034d79700 1 --2- 172.21.15.72:0/1471909210 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20300630e0 0x7f2030065700 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:08.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.964+0000 7f2034d79700 1 -- 172.21.15.72:0/1471909210 >> 172.21.15.72:0/1471909210 conn(0x7f2030062ba0 msgr2=0x7f20300c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:08.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.964+0000 7f2034d79700 1 -- 172.21.15.72:0/1471909210 shutdown_connections 2023-10-18T18:00:08.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.964+0000 7f2034d79700 1 -- 172.21.15.72:0/1471909210 wait complete. 2023-10-18T18:00:08.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.964+0000 7f2034d79700 1 Processor -- start 2023-10-18T18:00:08.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.964+0000 7f2034d79700 1 -- start start 2023-10-18T18:00:08.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2034d79700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20300630e0 0x7f2030159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:08.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2034d79700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20301513e0 0x7f203014ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:08.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2034d79700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20301517c0 0x7f2030159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:08.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2034d79700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203014d490 con 0x7f20300630e0 2023-10-18T18:00:08.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2034d79700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2030157f20 con 0x7f20301513e0 2023-10-18T18:00:08.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2034d79700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20300b95a0 con 0x7f20301517c0 2023-10-18T18:00:08.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20300630e0 0x7f2030159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:08.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20301517c0 0x7f2030159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:08.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2026ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20301513e0 0x7f203014ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:08.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20300630e0 0x7f2030159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:35340/0 (socket says 172.21.15.72:35340) 2023-10-18T18:00:08.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202f7fe700 1 -- 172.21.15.72:0/159959869 learned_addr learned my addr 172.21.15.72:0/159959869 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:08.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20301517c0 0x7f2030159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:43858/0 (socket says 172.21.15.72:43858) 2023-10-18T18:00:08.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202f7fe700 1 -- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20301517c0 msgr2=0x7f2030159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:08.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202f7fe700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20301517c0 0x7f2030159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:08.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202f7fe700 1 -- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20301513e0 msgr2=0x7f203014ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:08.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202f7fe700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20301513e0 0x7f203014ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:08.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202f7fe700 1 -- 172.21.15.72:0/159959869 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028009a20 con 0x7f20300630e0 2023-10-18T18:00:08.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f202f7fe700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20300630e0 0x7f2030159200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f202000c4d0 tx=0x7f2020007d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:08.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2027fff700 1 -- 172.21.15.72:0/159959869 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200072e0 con 0x7f20300630e0 2023-10-18T18:00:08.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2027fff700 1 -- 172.21.15.72:0/159959869 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020007440 con 0x7f20300630e0 2023-10-18T18:00:08.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20301596f0 con 0x7f20300630e0 2023-10-18T18:00:08.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2027fff700 1 -- 172.21.15.72:0/159959869 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200057a0 con 0x7f20300630e0 2023-10-18T18:00:08.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.968+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2030157c40 con 0x7f20300630e0 2023-10-18T18:00:08.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.972+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2030046f00 con 0x7f20300630e0 2023-10-18T18:00:08.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.972+0000 7f2027fff700 1 -- 172.21.15.72:0/159959869 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2020005b90 con 0x7f20300630e0 2023-10-18T18:00:08.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.972+0000 7f2027fff700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f201005a560 0x7f201005ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:08.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.972+0000 7f2027fff700 1 -- 172.21.15.72:0/159959869 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6494+0+0 (secure 0 0 0) 0x7f2020074240 con 0x7f20300630e0 2023-10-18T18:00:08.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.972+0000 7f2026ffd700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f201005a560 0x7f201005ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:08.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.976+0000 7f2026ffd700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f201005a560 0x7f201005ca20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f203015a4e0 tx=0x7f2018007bd0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:08.986 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:08.980+0000 7f2027fff700 1 -- 172.21.15.72:0/159959869 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f2020047870 con 0x7f20300630e0 2023-10-18T18:00:09.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.172+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f203015a7f0 con 0x7f20300630e0 2023-10-18T18:00:09.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.172+0000 7f2027fff700 1 -- 172.21.15.72:0/159959869 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f2020018080 con 0x7f20300630e0 2023-10-18T18:00:09.179 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:09.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f201005a560 msgr2=0x7f201005ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:09.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f201005a560 0x7f201005ca20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f203015a4e0 tx=0x7f2018007bd0).stop 2023-10-18T18:00:09.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20300630e0 msgr2=0x7f2030159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:09.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20300630e0 0x7f2030159200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f202000c4d0 tx=0x7f2020007d00).stop 2023-10-18T18:00:09.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 shutdown_connections 2023-10-18T18:00:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f201005a560 0x7f201005ca20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f20301517c0 0x7f2030159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20301513e0 0x7f203014ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 --2- 172.21.15.72:0/159959869 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20300630e0 0x7f2030159200 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:09.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 >> 172.21.15.72:0/159959869 conn(0x7f2030062ba0 msgr2=0x7f2030150790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 shutdown_connections 2023-10-18T18:00:09.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:09.176+0000 7f2034d79700 1 -- 172.21.15.72:0/159959869 wait complete. 2023-10-18T18:00:09.201 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:09.572 INFO:tasks.ceph.mds.e.smithi072.stderr:terminate called after throwing an instance of 'boost::wrapexcept' 2023-10-18T18:00:09.572 INFO:tasks.ceph.mds.e.smithi072.stderr: what(): boost::bad_get: failed value get using boost::get 2023-10-18T18:00:09.572 INFO:tasks.ceph.mds.e.smithi072.stderr:*** Caught signal (Aborted) ** 2023-10-18T18:00:09.572 INFO:tasks.ceph.mds.e.smithi072.stderr: in thread 7f46a2160700 thread_name:ms_dispatch 2023-10-18T18:00:09.573 INFO:tasks.ceph.mds.e.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T18:00:09.573 INFO:tasks.ceph.mds.e.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f46a73b6420] 2023-10-18T18:00:09.573 INFO:tasks.ceph.mds.e.smithi072.stderr: 2: gsignal() 2023-10-18T18:00:09.573 INFO:tasks.ceph.mds.e.smithi072.stderr: 3: abort() 2023-10-18T18:00:09.574 INFO:tasks.ceph.mds.e.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f46a725e911] 2023-10-18T18:00:09.574 INFO:tasks.ceph.mds.e.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f46a726a38c] 2023-10-18T18:00:09.574 INFO:tasks.ceph.mds.e.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f46a726a3f7] 2023-10-18T18:00:09.574 INFO:tasks.ceph.mds.e.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f46a726a6a9] 2023-10-18T18:00:09.574 INFO:tasks.ceph.mds.e.smithi072.stderr: 8: ceph-mds(+0x185107) [0x55dce76b2107] 2023-10-18T18:00:09.575 INFO:tasks.ceph.mds.e.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x55dce76b272d] 2023-10-18T18:00:09.575 INFO:tasks.ceph.mds.e.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55dce772409f] 2023-10-18T18:00:09.575 INFO:tasks.ceph.mds.e.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55dce7725a95] 2023-10-18T18:00:09.575 INFO:tasks.ceph.mds.e.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55dce76f0b10] 2023-10-18T18:00:09.575 INFO:tasks.ceph.mds.e.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55dce76f4abf] 2023-10-18T18:00:09.576 INFO:tasks.ceph.mds.e.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55dce76f5456] 2023-10-18T18:00:09.576 INFO:tasks.ceph.mds.e.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f46a78bc710] 2023-10-18T18:00:09.576 INFO:tasks.ceph.mds.e.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f46a78b9faf] 2023-10-18T18:00:09.576 INFO:tasks.ceph.mds.e.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f46a7988b21] 2023-10-18T18:00:09.576 INFO:tasks.ceph.mds.e.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f46a73aa609] 2023-10-18T18:00:09.576 INFO:tasks.ceph.mds.e.smithi072.stderr: 19: clone() 2023-10-18T18:00:09.577 INFO:tasks.ceph.mds.e.smithi072.stderr:2023-10-18T18:00:09.568+0000 7f46a2160700 -1 *** Caught signal (Aborted) ** 2023-10-18T18:00:09.577 INFO:tasks.ceph.mds.e.smithi072.stderr: in thread 7f46a2160700 thread_name:ms_dispatch 2023-10-18T18:00:09.577 INFO:tasks.ceph.mds.e.smithi072.stderr: 2023-10-18T18:00:09.578 INFO:tasks.ceph.mds.e.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T18:00:09.578 INFO:tasks.ceph.mds.e.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f46a73b6420] 2023-10-18T18:00:09.578 INFO:tasks.ceph.mds.e.smithi072.stderr: 2: gsignal() 2023-10-18T18:00:09.578 INFO:tasks.ceph.mds.e.smithi072.stderr: 3: abort() 2023-10-18T18:00:09.578 INFO:tasks.ceph.mds.e.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f46a725e911] 2023-10-18T18:00:09.579 INFO:tasks.ceph.mds.e.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f46a726a38c] 2023-10-18T18:00:09.579 INFO:tasks.ceph.mds.e.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f46a726a3f7] 2023-10-18T18:00:09.579 INFO:tasks.ceph.mds.e.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f46a726a6a9] 2023-10-18T18:00:09.579 INFO:tasks.ceph.mds.e.smithi072.stderr: 8: ceph-mds(+0x185107) [0x55dce76b2107] 2023-10-18T18:00:09.579 INFO:tasks.ceph.mds.e.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x55dce76b272d] 2023-10-18T18:00:09.580 INFO:tasks.ceph.mds.e.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55dce772409f] 2023-10-18T18:00:09.580 INFO:tasks.ceph.mds.e.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55dce7725a95] 2023-10-18T18:00:09.581 INFO:tasks.ceph.mds.e.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55dce76f0b10] 2023-10-18T18:00:09.581 INFO:tasks.ceph.mds.e.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55dce76f4abf] 2023-10-18T18:00:09.581 INFO:tasks.ceph.mds.e.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55dce76f5456] 2023-10-18T18:00:09.581 INFO:tasks.ceph.mds.e.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f46a78bc710] 2023-10-18T18:00:09.581 INFO:tasks.ceph.mds.e.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f46a78b9faf] 2023-10-18T18:00:09.582 INFO:tasks.ceph.mds.e.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f46a7988b21] 2023-10-18T18:00:09.582 INFO:tasks.ceph.mds.e.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f46a73aa609] 2023-10-18T18:00:09.582 INFO:tasks.ceph.mds.e.smithi072.stderr: 19: clone() 2023-10-18T18:00:09.583 INFO:tasks.ceph.mds.e.smithi072.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T18:00:09.583 INFO:tasks.ceph.mds.e.smithi072.stderr: 2023-10-18T18:00:09.583 INFO:tasks.ceph.mds.e.smithi072.stderr: 0> 2023-10-18T18:00:09.568+0000 7f46a2160700 -1 *** Caught signal (Aborted) ** 2023-10-18T18:00:09.583 INFO:tasks.ceph.mds.e.smithi072.stderr: in thread 7f46a2160700 thread_name:ms_dispatch 2023-10-18T18:00:09.583 INFO:tasks.ceph.mds.e.smithi072.stderr: 2023-10-18T18:00:09.584 INFO:tasks.ceph.mds.e.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T18:00:09.584 INFO:tasks.ceph.mds.e.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f46a73b6420] 2023-10-18T18:00:09.584 INFO:tasks.ceph.mds.e.smithi072.stderr: 2: gsignal() 2023-10-18T18:00:09.584 INFO:tasks.ceph.mds.e.smithi072.stderr: 3: abort() 2023-10-18T18:00:09.584 INFO:tasks.ceph.mds.e.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f46a725e911] 2023-10-18T18:00:09.584 INFO:tasks.ceph.mds.e.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f46a726a38c] 2023-10-18T18:00:09.585 INFO:tasks.ceph.mds.e.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f46a726a3f7] 2023-10-18T18:00:09.585 INFO:tasks.ceph.mds.e.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f46a726a6a9] 2023-10-18T18:00:09.585 INFO:tasks.ceph.mds.e.smithi072.stderr: 8: ceph-mds(+0x185107) [0x55dce76b2107] 2023-10-18T18:00:09.585 INFO:tasks.ceph.mds.e.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x55dce76b272d] 2023-10-18T18:00:09.585 INFO:tasks.ceph.mds.e.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55dce772409f] 2023-10-18T18:00:09.586 INFO:tasks.ceph.mds.e.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55dce7725a95] 2023-10-18T18:00:09.586 INFO:tasks.ceph.mds.e.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55dce76f0b10] 2023-10-18T18:00:09.586 INFO:tasks.ceph.mds.e.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55dce76f4abf] 2023-10-18T18:00:09.586 INFO:tasks.ceph.mds.e.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55dce76f5456] 2023-10-18T18:00:09.586 INFO:tasks.ceph.mds.e.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f46a78bc710] 2023-10-18T18:00:09.587 INFO:tasks.ceph.mds.e.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f46a78b9faf] 2023-10-18T18:00:09.587 INFO:tasks.ceph.mds.e.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f46a7988b21] 2023-10-18T18:00:09.587 INFO:tasks.ceph.mds.e.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f46a73aa609] 2023-10-18T18:00:09.587 INFO:tasks.ceph.mds.e.smithi072.stderr: 19: clone() 2023-10-18T18:00:09.587 INFO:tasks.ceph.mds.e.smithi072.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T18:00:09.587 INFO:tasks.ceph.mds.e.smithi072.stderr: 2023-10-18T18:00:09.588 INFO:tasks.ceph.mds.e.smithi072.stderr: 0> 2023-10-18T18:00:09.568+0000 7f46a2160700 -1 *** Caught signal (Aborted) ** 2023-10-18T18:00:09.588 INFO:tasks.ceph.mds.e.smithi072.stderr: in thread 7f46a2160700 thread_name:ms_dispatch 2023-10-18T18:00:09.588 INFO:tasks.ceph.mds.e.smithi072.stderr: 2023-10-18T18:00:09.589 INFO:tasks.ceph.mds.e.smithi072.stderr: ceph version 16.2.14-100-gb4ddd52c (b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5) pacific (stable) 2023-10-18T18:00:09.589 INFO:tasks.ceph.mds.e.smithi072.stderr: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f46a73b6420] 2023-10-18T18:00:09.589 INFO:tasks.ceph.mds.e.smithi072.stderr: 2: gsignal() 2023-10-18T18:00:09.589 INFO:tasks.ceph.mds.e.smithi072.stderr: 3: abort() 2023-10-18T18:00:09.589 INFO:tasks.ceph.mds.e.smithi072.stderr: 4: /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e911) [0x7f46a725e911] 2023-10-18T18:00:09.590 INFO:tasks.ceph.mds.e.smithi072.stderr: 5: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa38c) [0x7f46a726a38c] 2023-10-18T18:00:09.590 INFO:tasks.ceph.mds.e.smithi072.stderr: 6: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3f7) [0x7f46a726a3f7] 2023-10-18T18:00:09.590 INFO:tasks.ceph.mds.e.smithi072.stderr: 7: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa6a9) [0x7f46a726a6a9] 2023-10-18T18:00:09.590 INFO:tasks.ceph.mds.e.smithi072.stderr: 8: ceph-mds(+0x185107) [0x55dce76b2107] 2023-10-18T18:00:09.590 INFO:tasks.ceph.mds.e.smithi072.stderr: 9: ceph-mds(+0x18572d) [0x55dce76b272d] 2023-10-18T18:00:09.591 INFO:tasks.ceph.mds.e.smithi072.stderr: 10: (MDSRank::MDSRank(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x20f) [0x55dce772409f] 2023-10-18T18:00:09.591 INFO:tasks.ceph.mds.e.smithi072.stderr: 11: (MDSRankDispatcher::MDSRankDispatcher(int, std::__cxx11::basic_string, std::allocator >, ceph::fair_mutex&, std::shared_ptr&, CommonSafeTimer&, Beacon&, std::unique_ptr >&, Messenger*, MonClient*, MgrClient*, Context*, Context*, boost::asio::io_context&)+0x115) [0x55dce7725a95] 2023-10-18T18:00:09.591 INFO:tasks.ceph.mds.e.smithi072.stderr: 12: (MDSDaemon::handle_mds_map(boost::intrusive_ptr const&)+0x1530) [0x55dce76f0b10] 2023-10-18T18:00:09.591 INFO:tasks.ceph.mds.e.smithi072.stderr: 13: (MDSDaemon::handle_core_message(boost::intrusive_ptr const&)+0x3df) [0x55dce76f4abf] 2023-10-18T18:00:09.591 INFO:tasks.ceph.mds.e.smithi072.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x186) [0x55dce76f5456] 2023-10-18T18:00:09.592 INFO:tasks.ceph.mds.e.smithi072.stderr: 15: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f46a78bc710] 2023-10-18T18:00:09.592 INFO:tasks.ceph.mds.e.smithi072.stderr: 16: (DispatchQueue::entry()+0x58f) [0x7f46a78b9faf] 2023-10-18T18:00:09.592 INFO:tasks.ceph.mds.e.smithi072.stderr: 17: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f46a7988b21] 2023-10-18T18:00:09.592 INFO:tasks.ceph.mds.e.smithi072.stderr: 18: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f46a73aa609] 2023-10-18T18:00:09.592 INFO:tasks.ceph.mds.e.smithi072.stderr: 19: clone() 2023-10-18T18:00:09.592 INFO:tasks.ceph.mds.e.smithi072.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-10-18T18:00:09.593 INFO:tasks.ceph.mds.e.smithi072.stderr: 2023-10-18T18:00:09.687 INFO:tasks.ceph.mds.e.smithi072.stderr:daemon-helper: command crashed with signal 6 2023-10-18T18:00:11.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~100s 2023-10-18T18:00:11.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~46s 2023-10-18T18:00:11.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2023-10-18T18:00:11.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~26s 2023-10-18T18:00:11.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~13s 2023-10-18T18:00:11.135 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~59s 2023-10-18T18:00:12.202 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:12.365 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.360+0000 7f8f791b9700 1 Processor -- start 2023-10-18T18:00:12.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.360+0000 7f8f791b9700 1 -- start start 2023-10-18T18:00:12.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8f7414d170 0x7f8f7406a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:12.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 0x7f8f74065700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:12.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f74068910 con 0x7f8f74067d30 2023-10-18T18:00:12.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f74150c10 con 0x7f8f7414d170 2023-10-18T18:00:12.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74150780 con 0x7f8f74068180 2023-10-18T18:00:12.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f73fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8f7414d170 0x7f8f7406a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:12.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f789b8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 0x7f8f74065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:12.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f737fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f8f74067d30 0x7f8f740630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:47916/0 (socket says 172.21.15.72:47916) 2023-10-18T18:00:12.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f737fe700 1 -- 172.21.15.72:0/2457168888 learned_addr learned my addr 172.21.15.72:0/2457168888 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:12.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f789b8700 1 -- 172.21.15.72:0/2457168888 >> v2:172.21.15.179:3300/0 conn(0x7f8f7414d170 msgr2=0x7f8f7406a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:12.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f789b8700 1 --2- 172.21.15.72:0/2457168888 >> v2:172.21.15.179:3300/0 conn(0x7f8f7414d170 0x7f8f7406a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:12.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f789b8700 1 -- 172.21.15.72:0/2457168888 >> v1:172.21.15.179:6789/0 conn(0x7f8f74067d30 legacy=0x7f8f740630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:12.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f789b8700 1 -- 172.21.15.72:0/2457168888 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74068630 con 0x7f8f74068180 2023-10-18T18:00:12.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f73fff700 1 --2- 172.21.15.72:0/2457168888 >> v2:172.21.15.179:3300/0 conn(0x7f8f7414d170 0x7f8f7406a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:00:12.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f789b8700 1 --2- 172.21.15.72:0/2457168888 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 0x7f8f74065700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f8f6c009e60 tx=0x7f8f6c009780).ready entity=mon.2 client_cookie=df7e75a850c6b1f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:12.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f72ffd700 1 -- 172.21.15.72:0/2457168888 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00e070 con 0x7f8f74068180 2023-10-18T18:00:12.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f72ffd700 1 -- 172.21.15.72:0/2457168888 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c00bbf0 con 0x7f8f74068180 2023-10-18T18:00:12.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f72ffd700 1 -- 172.21.15.72:0/2457168888 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c012410 con 0x7f8f74068180 2023-10-18T18:00:12.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 -- 172.21.15.72:0/2457168888 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 msgr2=0x7f8f74065700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:12.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 --2- 172.21.15.72:0/2457168888 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 0x7f8f74065700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f8f6c009e60 tx=0x7f8f6c009780).stop 2023-10-18T18:00:12.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 -- 172.21.15.72:0/2457168888 shutdown_connections 2023-10-18T18:00:12.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 --2- 172.21.15.72:0/2457168888 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 0x7f8f74065700 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:12.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 --2- 172.21.15.72:0/2457168888 >> v2:172.21.15.179:3300/0 conn(0x7f8f7414d170 0x7f8f7406a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:12.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.364+0000 7f8f791b9700 1 -- 172.21.15.72:0/2457168888 >> 172.21.15.72:0/2457168888 conn(0x7f8f74062ba0 msgr2=0x7f8f740c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:12.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 -- 172.21.15.72:0/2457168888 shutdown_connections 2023-10-18T18:00:12.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 -- 172.21.15.72:0/2457168888 wait complete. 2023-10-18T18:00:12.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 Processor -- start 2023-10-18T18:00:12.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 -- start start 2023-10-18T18:00:12.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f74067d30 0x7f8f741510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:12.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 0x7f8f74156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:12.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f7414d170 0x7f8f740bf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:12.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74155e10 con 0x7f8f74067d30 2023-10-18T18:00:12.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74161a30 con 0x7f8f7414d170 2023-10-18T18:00:12.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.368+0000 7f8f791b9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74152070 con 0x7f8f74068180 2023-10-18T18:00:12.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f73fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f74067d30 0x7f8f741510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:12.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f789b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f7414d170 0x7f8f740bf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:12.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f73fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f74067d30 0x7f8f741510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55274/0 (socket says 172.21.15.72:55274) 2023-10-18T18:00:12.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f789b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f7414d170 0x7f8f740bf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:59682/0 (socket says 172.21.15.72:59682) 2023-10-18T18:00:12.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f73fff700 1 -- 172.21.15.72:0/3686349234 learned_addr learned my addr 172.21.15.72:0/3686349234 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:12.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f737fe700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 0x7f8f74156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:12.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f73fff700 1 -- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 msgr2=0x7f8f74156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:12.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f73fff700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 0x7f8f74156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:12.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f73fff700 1 -- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f7414d170 msgr2=0x7f8f740bf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:12.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f73fff700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f7414d170 0x7f8f740bf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:12.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f73fff700 1 -- 172.21.15.72:0/3686349234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f640092a0 con 0x7f8f74067d30 2023-10-18T18:00:12.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f73fff700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f74067d30 0x7f8f741510b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f8f6400ea50 tx=0x7f8f64009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:12.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f6bfff700 1 -- 172.21.15.72:0/3686349234 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400c8e0 con 0x7f8f74067d30 2023-10-18T18:00:12.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f6bfff700 1 -- 172.21.15.72:0/3686349234 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6400ca40 con 0x7f8f74067d30 2023-10-18T18:00:12.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f6bfff700 1 -- 172.21.15.72:0/3686349234 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400cdc0 con 0x7f8f74067d30 2023-10-18T18:00:12.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c009a20 con 0x7f8f74067d30 2023-10-18T18:00:12.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f741525c0 con 0x7f8f74067d30 2023-10-18T18:00:12.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.372+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f74161800 con 0x7f8f74067d30 2023-10-18T18:00:12.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.376+0000 7f8f6bfff700 1 -- 172.21.15.72:0/3686349234 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f8f64012600 con 0x7f8f74067d30 2023-10-18T18:00:12.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.376+0000 7f8f6bfff700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8f5405a510 0x7f8f5405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:12.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.376+0000 7f8f6bfff700 1 -- 172.21.15.72:0/3686349234 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f8f640750b0 con 0x7f8f74067d30 2023-10-18T18:00:12.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.376+0000 7f8f737fe700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8f5405a510 0x7f8f5405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:12.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.380+0000 7f8f737fe700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8f5405a510 0x7f8f5405c9d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f8f5c007fd0 tx=0x7f8f5c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:12.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.384+0000 7f8f6bfff700 1 -- 172.21.15.72:0/3686349234 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f8f74161800 con 0x7f8f74067d30 2023-10-18T18:00:12.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.576+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8f74161800 con 0x7f8f74067d30 2023-10-18T18:00:12.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.576+0000 7f8f6bfff700 1 -- 172.21.15.72:0/3686349234 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f8f74161800 con 0x7f8f74067d30 2023-10-18T18:00:12.584 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:12.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8f5405a510 msgr2=0x7f8f5405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:12.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8f5405a510 0x7f8f5405c9d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f8f5c007fd0 tx=0x7f8f5c009040).stop 2023-10-18T18:00:12.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f74067d30 msgr2=0x7f8f741510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:12.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f74067d30 0x7f8f741510b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f8f6400ea50 tx=0x7f8f64009060).stop 2023-10-18T18:00:12.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 shutdown_connections 2023-10-18T18:00:12.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f8f5405a510 0x7f8f5405c9d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:12.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f7414d170 0x7f8f740bf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:12.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8f74068180 0x7f8f74156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:12.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 --2- 172.21.15.72:0/3686349234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f74067d30 0x7f8f741510b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:12.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 >> 172.21.15.72:0/3686349234 conn(0x7f8f74062ba0 msgr2=0x7f8f7414ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:12.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 shutdown_connections 2023-10-18T18:00:12.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:12.580+0000 7f8f791b9700 1 -- 172.21.15.72:0/3686349234 wait complete. 2023-10-18T18:00:12.604 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:15.606 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:15.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.772+0000 7fed2b107700 1 Processor -- start 2023-10-18T18:00:15.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.772+0000 7fed2b107700 1 -- start start 2023-10-18T18:00:15.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2b107700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed2414d050 0x7fed24066c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:15.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2b107700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fed24063200 0x7fed240bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:15.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2b107700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed24150620 con 0x7fed24062dc0 2023-10-18T18:00:15.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2b107700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed24066890 con 0x7fed24063200 2023-10-18T18:00:15.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2b107700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed240bc6e0 con 0x7fed2414d050 2023-10-18T18:00:15.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2a105700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed2414d050 0x7fed24066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:15.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2a906700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fed24063200 0x7fed240bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:15.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2a105700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed2414d050 0x7fed24066c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:45506/0 (socket says 172.21.15.72:45506) 2023-10-18T18:00:15.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2a105700 1 -- 172.21.15.72:0/1312583167 learned_addr learned my addr 172.21.15.72:0/1312583167 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:15.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2a105700 1 -- 172.21.15.72:0/1312583167 >> v2:172.21.15.179:3300/0 conn(0x7fed24063200 msgr2=0x7fed240bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:15.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2a105700 1 --2- 172.21.15.72:0/1312583167 >> v2:172.21.15.179:3300/0 conn(0x7fed24063200 0x7fed240bd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:15.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2a105700 1 -- 172.21.15.72:0/1312583167 >> v1:172.21.15.72:6789/0 conn(0x7fed24062dc0 legacy=0x7fed2414fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:15.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2a105700 1 -- 172.21.15.72:0/1312583167 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed24150900 con 0x7fed2414d050 2023-10-18T18:00:15.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed2a105700 1 --2- 172.21.15.72:0/1312583167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed2414d050 0x7fed24066c40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fed20009f20 tx=0x7fed20009840).ready entity=mon.2 client_cookie=e9aedc2a8739259f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:15.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.776+0000 7fed29103700 1 -- 172.21.15.72:0/1312583167 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2000e070 con 0x7fed2414d050 2023-10-18T18:00:15.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed29103700 1 -- 172.21.15.72:0/1312583167 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2000b9d0 con 0x7fed2414d050 2023-10-18T18:00:15.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed29103700 1 -- 172.21.15.72:0/1312583167 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2000bd50 con 0x7fed2414d050 2023-10-18T18:00:15.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed2b107700 1 -- 172.21.15.72:0/1312583167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed2414d050 msgr2=0x7fed24066c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:15.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed2b107700 1 --2- 172.21.15.72:0/1312583167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed2414d050 0x7fed24066c40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fed20009f20 tx=0x7fed20009840).stop 2023-10-18T18:00:15.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed2b107700 1 -- 172.21.15.72:0/1312583167 shutdown_connections 2023-10-18T18:00:15.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed2b107700 1 --2- 172.21.15.72:0/1312583167 >> v2:172.21.15.179:3300/0 conn(0x7fed24063200 0x7fed240bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:15.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed2b107700 1 --2- 172.21.15.72:0/1312583167 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed2414d050 0x7fed24066c40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:15.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed2b107700 1 -- 172.21.15.72:0/1312583167 >> 172.21.15.72:0/1312583167 conn(0x7fed240c2780 msgr2=0x7fed240c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:15.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed2b107700 1 -- 172.21.15.72:0/1312583167 shutdown_connections 2023-10-18T18:00:15.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.780+0000 7fed2b107700 1 -- 172.21.15.72:0/1312583167 wait complete. 2023-10-18T18:00:15.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2b107700 1 Processor -- start 2023-10-18T18:00:15.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2b107700 1 -- start start 2023-10-18T18:00:15.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2b107700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed24062dc0 0x7fed24150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:15.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2b107700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed24063200 0x7fed240b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:15.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2b107700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2414d050 0x7fed2414f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:15.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2b107700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240b5e20 con 0x7fed24063200 2023-10-18T18:00:15.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2b107700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed241513b0 con 0x7fed2414d050 2023-10-18T18:00:15.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2b107700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed24151620 con 0x7fed24062dc0 2023-10-18T18:00:15.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2a906700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2414d050 0x7fed2414f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:15.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2a105700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed24062dc0 0x7fed24150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:15.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2a906700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2414d050 0x7fed2414f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:59692/0 (socket says 172.21.15.72:59692) 2023-10-18T18:00:15.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2a105700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed24062dc0 0x7fed24150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:45522/0 (socket says 172.21.15.72:45522) 2023-10-18T18:00:15.794 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed2a906700 1 -- 172.21.15.72:0/2690279217 learned_addr learned my addr 172.21.15.72:0/2690279217 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:15.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.784+0000 7fed29904700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed24063200 0x7fed240b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:15.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed2a906700 1 -- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed24062dc0 msgr2=0x7fed24150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:15.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed2a906700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed24062dc0 0x7fed24150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:15.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed2a906700 1 -- 172.21.15.72:0/2690279217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed24063200 msgr2=0x7fed240b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:15.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed2a906700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed24063200 0x7fed240b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:15.795 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed2a906700 1 -- 172.21.15.72:0/2690279217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed20009ae0 con 0x7fed2414d050 2023-10-18T18:00:15.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed29904700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed24063200 0x7fed240b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:00:15.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed2a906700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2414d050 0x7fed2414f9b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fed1c00e900 tx=0x7fed1c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:15.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed1a7fc700 1 -- 172.21.15.72:0/2690279217 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c012600 con 0x7fed2414d050 2023-10-18T18:00:15.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed1a7fc700 1 -- 172.21.15.72:0/2690279217 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c012760 con 0x7fed2414d050 2023-10-18T18:00:15.796 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed24151900 con 0x7fed2414d050 2023-10-18T18:00:15.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed1a7fc700 1 -- 172.21.15.72:0/2690279217 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c012ae0 con 0x7fed2414d050 2023-10-18T18:00:15.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed240b57d0 con 0x7fed2414d050 2023-10-18T18:00:15.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed240b5fd0 con 0x7fed2414d050 2023-10-18T18:00:15.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.788+0000 7fed1a7fc700 1 -- 172.21.15.72:0/2690279217 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fed1c01b400 con 0x7fed2414d050 2023-10-18T18:00:15.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.792+0000 7fed1a7fc700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fed0805a510 0x7fed0805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:15.798 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.792+0000 7fed1a7fc700 1 -- 172.21.15.72:0/2690279217 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fed1c074eb0 con 0x7fed2414d050 2023-10-18T18:00:15.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.792+0000 7fed2a105700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fed0805a510 0x7fed0805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:15.799 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.792+0000 7fed2a105700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fed0805a510 0x7fed0805c9d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fed20009f20 tx=0x7fed20009840).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:15.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.800+0000 7fed1a7fc700 1 -- 172.21.15.72:0/2690279217 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fed240b5fd0 con 0x7fed2414d050 2023-10-18T18:00:16.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:15.996+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fed240b5f90 con 0x7fed2414d050 2023-10-18T18:00:16.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed1a7fc700 1 -- 172.21.15.72:0/2690279217 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fed1c016020 con 0x7fed2414d050 2023-10-18T18:00:16.005 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:16.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fed0805a510 msgr2=0x7fed0805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:16.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fed0805a510 0x7fed0805c9d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fed20009f20 tx=0x7fed20009840).stop 2023-10-18T18:00:16.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2414d050 msgr2=0x7fed2414f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:16.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2414d050 0x7fed2414f9b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fed1c00e900 tx=0x7fed1c009060).stop 2023-10-18T18:00:16.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 shutdown_connections 2023-10-18T18:00:16.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fed0805a510 0x7fed0805c9d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:16.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed2414d050 0x7fed2414f9b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:16.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed24063200 0x7fed240b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:16.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 --2- 172.21.15.72:0/2690279217 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fed24062dc0 0x7fed24150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:16.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 >> 172.21.15.72:0/2690279217 conn(0x7fed240c2780 msgr2=0x7fed240b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:16.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 shutdown_connections 2023-10-18T18:00:16.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:16.000+0000 7fed2b107700 1 -- 172.21.15.72:0/2690279217 wait complete. 2023-10-18T18:00:16.026 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:17.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~106s 2023-10-18T18:00:17.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~52s 2023-10-18T18:00:17.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~6s 2023-10-18T18:00:17.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~32s 2023-10-18T18:00:17.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~19s 2023-10-18T18:00:17.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~65s 2023-10-18T18:00:19.027 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:19.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f06116f1700 1 Processor -- start 2023-10-18T18:00:19.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f06116f1700 1 -- start start 2023-10-18T18:00:19.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f06116f1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f060c14b1d0 0x7f060c14e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:19.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f06116f1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c14dd30 con 0x7f060c14b5b0 2023-10-18T18:00:19.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f06116f1700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c062430 con 0x7f060c0bf8d0 2023-10-18T18:00:19.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f06116f1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060c062710 con 0x7f060c14b1d0 2023-10-18T18:00:19.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f06027fc700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f060c14b5b0 0x7f060c0bcd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:49052/0 (socket says 172.21.15.72:49052) 2023-10-18T18:00:19.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f06027fc700 1 -- 172.21.15.72:0/1052345355 learned_addr learned my addr 172.21.15.72:0/1052345355 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:19.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f060affd700 1 --2- 172.21.15.72:0/1052345355 >> v2:172.21.15.179:3300/0 conn(0x7f060c14b1d0 0x7f060c14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:19.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f060a7fc700 1 -- 172.21.15.72:0/1052345355 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305808772 0 0) 0x7f060c14dd30 con 0x7f060c14b5b0 2023-10-18T18:00:19.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f060affd700 1 -- 172.21.15.72:0/1052345355 >> v1:172.21.15.179:6789/0 conn(0x7f060c0bf8d0 legacy=0x7f060c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:19.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f060affd700 1 -- 172.21.15.72:0/1052345355 >> v1:172.21.15.72:6789/0 conn(0x7f060c14b5b0 legacy=0x7f060c0bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:19.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.187+0000 7f060affd700 1 -- 172.21.15.72:0/1052345355 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0629f0 con 0x7f060c14b1d0 2023-10-18T18:00:19.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f060affd700 1 --2- 172.21.15.72:0/1052345355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c14b1d0 0x7f060c14e510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f05fc009e60 tx=0x7f05fc009780).ready entity=mon.1 client_cookie=2b4ffca47367e4c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:19.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f060a7fc700 1 -- 172.21.15.72:0/1052345355 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00e070 con 0x7f060c14b1d0 2023-10-18T18:00:19.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f060a7fc700 1 -- 172.21.15.72:0/1052345355 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc00bc30 con 0x7f060c14b1d0 2023-10-18T18:00:19.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f060a7fc700 1 -- 172.21.15.72:0/1052345355 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc012410 con 0x7f060c14b1d0 2023-10-18T18:00:19.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f06116f1700 1 -- 172.21.15.72:0/1052345355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c14b1d0 msgr2=0x7f060c14e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:19.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f06116f1700 1 --2- 172.21.15.72:0/1052345355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c14b1d0 0x7f060c14e510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f05fc009e60 tx=0x7f05fc009780).stop 2023-10-18T18:00:19.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f06116f1700 1 -- 172.21.15.72:0/1052345355 shutdown_connections 2023-10-18T18:00:19.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f06116f1700 1 --2- 172.21.15.72:0/1052345355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c14b1d0 0x7f060c14e510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:19.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f06116f1700 1 -- 172.21.15.72:0/1052345355 >> 172.21.15.72:0/1052345355 conn(0x7f060c0699e0 msgr2=0x7f060c069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:19.200 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f06116f1700 1 -- 172.21.15.72:0/1052345355 shutdown_connections 2023-10-18T18:00:19.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f06116f1700 1 -- 172.21.15.72:0/1052345355 wait complete. 2023-10-18T18:00:19.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.191+0000 7f06116f1700 1 Processor -- start 2023-10-18T18:00:19.201 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06116f1700 1 -- start start 2023-10-18T18:00:19.202 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06116f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f060c0bf8d0 0x7f060c0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:19.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06116f1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c14b1d0 0x7f060c0c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:19.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06116f1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f060c0c1f40 0x7f060c1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:19.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06116f1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c160350 con 0x7f060c0c1f40 2023-10-18T18:00:19.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06116f1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0c03a0 con 0x7f060c14b1d0 2023-10-18T18:00:19.203 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06116f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0bc040 con 0x7f060c0bf8d0 2023-10-18T18:00:19.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f060c0c1f40 0x7f060c1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:19.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060affd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f060c0bf8d0 0x7f060c0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:19.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060affd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f060c0bf8d0 0x7f060c0b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:45532/0 (socket says 172.21.15.72:45532) 2023-10-18T18:00:19.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060affd700 1 -- 172.21.15.72:0/959382123 learned_addr learned my addr 172.21.15.72:0/959382123 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:19.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06027fc700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c14b1d0 0x7f060c0c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:19.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060affd700 1 -- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c14b1d0 msgr2=0x7f060c0c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:19.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060affd700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c14b1d0 0x7f060c0c1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:19.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060affd700 1 -- 172.21.15.72:0/959382123 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f060c0c1f40 msgr2=0x7f060c1607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:19.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060affd700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f060c0c1f40 0x7f060c1607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:19.206 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060affd700 1 -- 172.21.15.72:0/959382123 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc009a20 con 0x7f060c0bf8d0 2023-10-18T18:00:19.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060bfff700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f060c0c1f40 0x7f060c1607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:00:19.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f060affd700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f060c0bf8d0 0x7f060c0b5d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f05fc00b6c0 tx=0x7f05fc009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:19.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06037fe700 1 -- 172.21.15.72:0/959382123 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc001090 con 0x7f060c0bf8d0 2023-10-18T18:00:19.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06037fe700 1 -- 172.21.15.72:0/959382123 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc01d630 con 0x7f060c0bf8d0 2023-10-18T18:00:19.207 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06037fe700 1 -- 172.21.15.72:0/959382123 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc01d9b0 con 0x7f060c0bf8d0 2023-10-18T18:00:19.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c0bc320 con 0x7f060c0bf8d0 2023-10-18T18:00:19.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.195+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c0bc8e0 con 0x7f060c0bf8d0 2023-10-18T18:00:19.208 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.199+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c06c190 con 0x7f060c0bf8d0 2023-10-18T18:00:19.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.199+0000 7f06037fe700 1 -- 172.21.15.72:0/959382123 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f05fc027460 con 0x7f060c0bf8d0 2023-10-18T18:00:19.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.203+0000 7f06037fe700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f05ec05a510 0x7f05ec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:19.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.203+0000 7f06037fe700 1 -- 172.21.15.72:0/959382123 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f05fc076cb0 con 0x7f060c0bf8d0 2023-10-18T18:00:19.209 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.203+0000 7f06027fc700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f05ec05a510 0x7f05ec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:19.210 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.203+0000 7f06027fc700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f05ec05a510 0x7f05ec05c9d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f05f4007fd0 tx=0x7f05f4009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:19.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.211+0000 7f06037fe700 1 -- 172.21.15.72:0/959382123 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f05fc04a1e0 con 0x7f060c0bf8d0 2023-10-18T18:00:19.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.403+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f060c062cd0 con 0x7f060c0bf8d0 2023-10-18T18:00:19.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.403+0000 7f06037fe700 1 -- 172.21.15.72:0/959382123 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f05fc04d7c0 con 0x7f060c0bf8d0 2023-10-18T18:00:19.410 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:19.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f05ec05a510 msgr2=0x7f05ec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:19.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f05ec05a510 0x7f05ec05c9d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f05f4007fd0 tx=0x7f05f4009040).stop 2023-10-18T18:00:19.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f060c0bf8d0 msgr2=0x7f060c0b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:19.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f060c0bf8d0 0x7f060c0b5d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f05fc00b6c0 tx=0x7f05fc009780).stop 2023-10-18T18:00:19.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 shutdown_connections 2023-10-18T18:00:19.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f05ec05a510 0x7f05ec05c9d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:19.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f060c0c1f40 0x7f060c1607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:19.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f060c14b1d0 0x7f060c0c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:19.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 --2- 172.21.15.72:0/959382123 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f060c0bf8d0 0x7f060c0b5d00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:19.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 >> 172.21.15.72:0/959382123 conn(0x7f060c0699e0 msgr2=0x7f060c14a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:19.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 shutdown_connections 2023-10-18T18:00:19.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:19.407+0000 7f06116f1700 1 -- 172.21.15.72:0/959382123 wait complete. 2023-10-18T18:00:19.434 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:22.436 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:22.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.587+0000 7feb4305b700 1 Processor -- start 2023-10-18T18:00:22.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb4305b700 1 -- start start 2023-10-18T18:00:22.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb4305b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feb3c0b9690 0x7feb3c0b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:22.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb4305b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c14dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:22.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb4305b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c0bc480 con 0x7feb3c0bca50 2023-10-18T18:00:22.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb4305b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3c065190 con 0x7feb3c0b9690 2023-10-18T18:00:22.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb4305b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0bb920 con 0x7feb3c065d30 2023-10-18T18:00:22.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb40df7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feb3c0b9690 0x7feb3c0b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:22.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb40df7700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7feb3c0b9690 0x7feb3c0b9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:36002/0 (socket says 172.21.15.72:36002) 2023-10-18T18:00:22.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb415f8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c14dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:22.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb40df7700 1 -- 172.21.15.72:0/4125861874 learned_addr learned my addr 172.21.15.72:0/4125861874 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:22.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb337fe700 1 -- 172.21.15.72:0/4125861874 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384214842 0 0) 0x7feb3c0bc480 con 0x7feb3c0bca50 2023-10-18T18:00:22.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb337fe700 1 -- 172.21.15.72:0/4125861874 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb240033f0 con 0x7feb3c0bca50 2023-10-18T18:00:22.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb40df7700 1 -- 172.21.15.72:0/4125861874 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 msgr2=0x7feb3c14dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:22.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb40df7700 1 --2- 172.21.15.72:0/4125861874 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c14dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:22.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb40df7700 1 -- 172.21.15.72:0/4125861874 >> v1:172.21.15.72:6789/0 conn(0x7feb3c0bca50 legacy=0x7feb3c065620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:22.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb40df7700 1 -- 172.21.15.72:0/4125861874 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c0bbbe0 con 0x7feb3c0b9690 2023-10-18T18:00:22.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.591+0000 7feb40df7700 1 --2- 172.21.15.72:0/4125861874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb3c0b9690 0x7feb3c0b9a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7feb38006fd0 tx=0x7feb380090f0).ready entity=mon.1 client_cookie=957fa09506130ef6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:22.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb337fe700 1 -- 172.21.15.72:0/4125861874 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800f510 con 0x7feb3c0b9690 2023-10-18T18:00:22.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb337fe700 1 -- 172.21.15.72:0/4125861874 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3800f670 con 0x7feb3c0b9690 2023-10-18T18:00:22.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb337fe700 1 -- 172.21.15.72:0/4125861874 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800f9f0 con 0x7feb3c0b9690 2023-10-18T18:00:22.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb4305b700 1 -- 172.21.15.72:0/4125861874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb3c0b9690 msgr2=0x7feb3c0b9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:22.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb4305b700 1 --2- 172.21.15.72:0/4125861874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb3c0b9690 0x7feb3c0b9a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7feb38006fd0 tx=0x7feb380090f0).stop 2023-10-18T18:00:22.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb4305b700 1 -- 172.21.15.72:0/4125861874 shutdown_connections 2023-10-18T18:00:22.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb4305b700 1 --2- 172.21.15.72:0/4125861874 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c14dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb4305b700 1 --2- 172.21.15.72:0/4125861874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb3c0b9690 0x7feb3c0b9a70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb4305b700 1 -- 172.21.15.72:0/4125861874 >> 172.21.15.72:0/4125861874 conn(0x7feb3c0620d0 msgr2=0x7feb3c0624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb4305b700 1 -- 172.21.15.72:0/4125861874 shutdown_connections 2023-10-18T18:00:22.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb4305b700 1 -- 172.21.15.72:0/4125861874 wait complete. 2023-10-18T18:00:22.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.595+0000 7feb4305b700 1 Processor -- start 2023-10-18T18:00:22.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb4305b700 1 -- start start 2023-10-18T18:00:22.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb4305b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:22.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb4305b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb3c0b9690 0x7feb3c163a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:22.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb4305b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb3c0656d0 0x7feb3c068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:22.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb4305b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c14d8b0 con 0x7feb3c0656d0 2023-10-18T18:00:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb4305b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c068950 con 0x7feb3c0b9690 2023-10-18T18:00:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb4305b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c164720 con 0x7feb3c065d30 2023-10-18T18:00:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb40df7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:22.608 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb40df7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:32926/0 (socket says 172.21.15.72:32926) 2023-10-18T18:00:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb415f8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb3c0656d0 0x7feb3c068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb40df7700 1 -- 172.21.15.72:0/2776776918 learned_addr learned my addr 172.21.15.72:0/2776776918 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:22.609 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb33fff700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb3c0b9690 0x7feb3c163a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb40df7700 1 -- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb3c0b9690 msgr2=0x7feb3c163a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb40df7700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb3c0b9690 0x7feb3c163a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb40df7700 1 -- 172.21.15.72:0/2776776918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb3c0656d0 msgr2=0x7feb3c068de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb40df7700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb3c0656d0 0x7feb3c068de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:22.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb40df7700 1 -- 172.21.15.72:0/2776776918 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb38009370 con 0x7feb3c065d30 2023-10-18T18:00:22.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb40df7700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c141480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7feb38009df0 tx=0x7feb38017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:22.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb30ff9700 1 -- 172.21.15.72:0/2776776918 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800fe80 con 0x7feb3c065d30 2023-10-18T18:00:22.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb30ff9700 1 -- 172.21.15.72:0/2776776918 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3800f7e0 con 0x7feb3c065d30 2023-10-18T18:00:22.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c068060 con 0x7feb3c065d30 2023-10-18T18:00:22.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb30ff9700 1 -- 172.21.15.72:0/2776776918 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3801c650 con 0x7feb3c065d30 2023-10-18T18:00:22.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.599+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c068300 con 0x7feb3c065d30 2023-10-18T18:00:22.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.603+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3c06c190 con 0x7feb3c065d30 2023-10-18T18:00:22.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.603+0000 7feb30ff9700 1 -- 172.21.15.72:0/2776776918 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7feb3801ca40 con 0x7feb3c065d30 2023-10-18T18:00:22.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.603+0000 7feb30ff9700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7feb2805a510 0x7feb2805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:22.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.607+0000 7feb30ff9700 1 -- 172.21.15.72:0/2776776918 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7feb38076530 con 0x7feb3c065d30 2023-10-18T18:00:22.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.607+0000 7feb33fff700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7feb2805a510 0x7feb2805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:22.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.607+0000 7feb33fff700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7feb2805a510 0x7feb2805c9d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7feb3c06a110 tx=0x7feb2c007bd0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:22.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.615+0000 7feb30ff9700 1 -- 172.21.15.72:0/2776776918 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7feb38049ac0 con 0x7feb3c065d30 2023-10-18T18:00:22.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.823+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7feb3c0bbec0 con 0x7feb3c065d30 2023-10-18T18:00:22.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.823+0000 7feb30ff9700 1 -- 172.21.15.72:0/2776776918 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7feb3802a080 con 0x7feb3c065d30 2023-10-18T18:00:22.832 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:22.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.827+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7feb2805a510 msgr2=0x7feb2805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:22.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.827+0000 7feb4305b700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7feb2805a510 0x7feb2805c9d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7feb3c06a110 tx=0x7feb2c007bd0).stop 2023-10-18T18:00:22.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 msgr2=0x7feb3c141480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:22.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c141480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7feb38009df0 tx=0x7feb38017000).stop 2023-10-18T18:00:22.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 shutdown_connections 2023-10-18T18:00:22.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7feb2805a510 0x7feb2805c9d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:22.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb3c0656d0 0x7feb3c068de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:22.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb3c0b9690 0x7feb3c163a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:22.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 --2- 172.21.15.72:0/2776776918 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7feb3c065d30 0x7feb3c141480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:22.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 >> 172.21.15.72:0/2776776918 conn(0x7feb3c0620d0 msgr2=0x7feb3c0be450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:22.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 shutdown_connections 2023-10-18T18:00:22.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:22.831+0000 7feb4305b700 1 -- 172.21.15.72:0/2776776918 wait complete. 2023-10-18T18:00:22.859 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:23.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~113s 2023-10-18T18:00:23.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~58s 2023-10-18T18:00:23.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~13s 2023-10-18T18:00:23.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~38s 2023-10-18T18:00:23.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~25s 2023-10-18T18:00:23.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~72s 2023-10-18T18:00:25.860 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:26.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595eae4700 1 Processor -- start 2023-10-18T18:00:26.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595eae4700 1 -- start start 2023-10-18T18:00:26.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595eae4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 0x7f595806a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:26.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595eae4700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f59580630e0 0x7f5958065700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:26.027 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595eae4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5958154b00 con 0x7f59581513e0 2023-10-18T18:00:26.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595eae4700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5958069380 con 0x7f59580630e0 2023-10-18T18:00:26.028 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595eae4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5958154f90 con 0x7f59581517c0 2023-10-18T18:00:26.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595d2e1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 0x7f595806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:26.029 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595e2e3700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f59580630e0 0x7f5958065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:26.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595dae2700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f59581513e0 0x7f5958067d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:60878/0 (socket says 172.21.15.72:60878) 2023-10-18T18:00:26.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595dae2700 1 -- 172.21.15.72:0/1092608605 learned_addr learned my addr 172.21.15.72:0/1092608605 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:26.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595d2e1700 1 -- 172.21.15.72:0/1092608605 >> v2:172.21.15.179:3300/0 conn(0x7f59580630e0 msgr2=0x7f5958065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:26.030 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.019+0000 7f595cae0700 1 -- 172.21.15.72:0/1092608605 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403003217 0 0) 0x7f5958154b00 con 0x7f59581513e0 2023-10-18T18:00:26.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595d2e1700 1 --2- 172.21.15.72:0/1092608605 >> v2:172.21.15.179:3300/0 conn(0x7f59580630e0 0x7f5958065700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:26.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595d2e1700 1 -- 172.21.15.72:0/1092608605 >> v1:172.21.15.179:6789/0 conn(0x7f59581513e0 legacy=0x7f5958067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:26.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595d2e1700 1 -- 172.21.15.72:0/1092608605 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958068570 con 0x7f59581517c0 2023-10-18T18:00:26.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595d2e1700 1 --2- 172.21.15.72:0/1092608605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 0x7f595806a040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5948009760 tx=0x7f5948009060).ready entity=mon.2 client_cookie=7e1a47c3e654994 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:26.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595cae0700 1 -- 172.21.15.72:0/1092608605 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800f510 con 0x7f59581517c0 2023-10-18T18:00:26.031 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595cae0700 1 -- 172.21.15.72:0/1092608605 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594800f670 con 0x7f59581517c0 2023-10-18T18:00:26.032 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595cae0700 1 -- 172.21.15.72:0/1092608605 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800f9f0 con 0x7f59581517c0 2023-10-18T18:00:26.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595eae4700 1 -- 172.21.15.72:0/1092608605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 msgr2=0x7f595806a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:26.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595eae4700 1 --2- 172.21.15.72:0/1092608605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 0x7f595806a040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5948009760 tx=0x7f5948009060).stop 2023-10-18T18:00:26.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595eae4700 1 -- 172.21.15.72:0/1092608605 shutdown_connections 2023-10-18T18:00:26.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595eae4700 1 --2- 172.21.15.72:0/1092608605 >> v2:172.21.15.179:3300/0 conn(0x7f59580630e0 0x7f5958065700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:26.033 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595eae4700 1 --2- 172.21.15.72:0/1092608605 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 0x7f595806a040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:26.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595eae4700 1 -- 172.21.15.72:0/1092608605 >> 172.21.15.72:0/1092608605 conn(0x7f5958062ba0 msgr2=0x7f59580c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:26.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595eae4700 1 -- 172.21.15.72:0/1092608605 shutdown_connections 2023-10-18T18:00:26.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.023+0000 7f595eae4700 1 -- 172.21.15.72:0/1092608605 wait complete. 2023-10-18T18:00:26.034 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595eae4700 1 Processor -- start 2023-10-18T18:00:26.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595eae4700 1 -- start start 2023-10-18T18:00:26.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595eae4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59580630e0 0x7f595814c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:26.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595eae4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59581513e0 0x7f5958155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:26.037 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595eae4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 0x7f5958157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:26.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595eae4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5958157050 con 0x7f59581513e0 2023-10-18T18:00:26.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595eae4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59581590f0 con 0x7f59580630e0 2023-10-18T18:00:26.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595eae4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59581567d0 con 0x7f59581517c0 2023-10-18T18:00:26.038 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595d2e1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59581513e0 0x7f5958155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:26.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595dae2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59580630e0 0x7f595814c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:26.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595d2e1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59581513e0 0x7f5958155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44296/0 (socket says 172.21.15.72:44296) 2023-10-18T18:00:26.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595d2e1700 1 -- 172.21.15.72:0/2606278280 learned_addr learned my addr 172.21.15.72:0/2606278280 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:26.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595e2e3700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 0x7f5958157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:26.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595d2e1700 1 -- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 msgr2=0x7f5958157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:26.039 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595d2e1700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 0x7f5958157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595d2e1700 1 -- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59580630e0 msgr2=0x7f595814c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595d2e1700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59580630e0 0x7f595814c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.027+0000 7f595d2e1700 1 -- 172.21.15.72:0/2606278280 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480092a0 con 0x7f59581513e0 2023-10-18T18:00:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.031+0000 7f595d2e1700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59581513e0 0x7f5958155e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5948009e00 tx=0x7f5948017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:26.040 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.031+0000 7f594dffb700 1 -- 172.21.15.72:0/2606278280 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800f7e0 con 0x7f59581513e0 2023-10-18T18:00:26.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.031+0000 7f594dffb700 1 -- 172.21.15.72:0/2606278280 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594800f940 con 0x7f59581513e0 2023-10-18T18:00:26.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.031+0000 7f594dffb700 1 -- 172.21.15.72:0/2606278280 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594801b660 con 0x7f59581513e0 2023-10-18T18:00:26.041 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.031+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5958158710 con 0x7f59581513e0 2023-10-18T18:00:26.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.031+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5958156d70 con 0x7f59581513e0 2023-10-18T18:00:26.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.031+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5958046f00 con 0x7f59581513e0 2023-10-18T18:00:26.042 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.035+0000 7f594dffb700 1 -- 172.21.15.72:0/2606278280 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f594801ba50 con 0x7f59581513e0 2023-10-18T18:00:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.035+0000 7f594dffb700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f593c05a540 0x7f593c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.035+0000 7f594dffb700 1 -- 172.21.15.72:0/2606278280 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f5948076440 con 0x7f59581513e0 2023-10-18T18:00:26.043 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.035+0000 7f595dae2700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f593c05a540 0x7f593c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:26.044 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.035+0000 7f595dae2700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f593c05a540 0x7f593c05ca00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f59540011e0 tx=0x7f5954007bb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:26.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.043+0000 7f594dffb700 1 -- 172.21.15.72:0/2606278280 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f5948049990 con 0x7f59581513e0 2023-10-18T18:00:26.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.235+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f59581541d0 con 0x7f59581513e0 2023-10-18T18:00:26.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.235+0000 7f594dffb700 1 -- 172.21.15.72:0/2606278280 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f594804cfb0 con 0x7f59581513e0 2023-10-18T18:00:26.242 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:26.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f593c05a540 msgr2=0x7f593c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:26.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f593c05a540 0x7f593c05ca00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f59540011e0 tx=0x7f5954007bb0).stop 2023-10-18T18:00:26.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59581513e0 msgr2=0x7f5958155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:26.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59581513e0 0x7f5958155e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5948009e00 tx=0x7f5948017000).stop 2023-10-18T18:00:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 shutdown_connections 2023-10-18T18:00:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f593c05a540 0x7f593c05ca00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59581517c0 0x7f5958157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59581513e0 0x7f5958155e80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:26.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 --2- 172.21.15.72:0/2606278280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59580630e0 0x7f595814c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:26.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 >> 172.21.15.72:0/2606278280 conn(0x7f5958062ba0 msgr2=0x7f59580bcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:26.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 shutdown_connections 2023-10-18T18:00:26.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:26.239+0000 7f595eae4700 1 -- 172.21.15.72:0/2606278280 wait complete. 2023-10-18T18:00:26.262 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:26.620 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:00:26.628 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:00:29.263 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:29.431 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.427+0000 7fdfe58f1700 1 Processor -- start 2023-10-18T18:00:29.432 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.427+0000 7fdfe58f1700 1 -- start start 2023-10-18T18:00:29.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfe58f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b5b0 0x7fdfe00bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:29.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfe58f1700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe014dd30 con 0x7fdfe014b1d0 2023-10-18T18:00:29.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfe58f1700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe0062430 con 0x7fdfe00bf8d0 2023-10-18T18:00:29.434 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfe58f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0062710 con 0x7fdfe014b5b0 2023-10-18T18:00:29.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfdeffd700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fdfe014b1d0 0x7fdfe014e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:43396/0 (socket says 172.21.15.72:43396) 2023-10-18T18:00:29.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfdeffd700 1 -- 172.21.15.72:0/2897443114 learned_addr learned my addr 172.21.15.72:0/2897443114 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:29.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfd67fc700 1 --2- 172.21.15.72:0/2897443114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b5b0 0x7fdfe00bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:29.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfde7fc700 1 -- 172.21.15.72:0/2897443114 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968265728 0 0) 0x7fdfe0062430 con 0x7fdfe00bf8d0 2023-10-18T18:00:29.436 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfde7fc700 1 -- 172.21.15.72:0/2897443114 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfcc0033f0 con 0x7fdfe00bf8d0 2023-10-18T18:00:29.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfde7fc700 1 -- 172.21.15.72:0/2897443114 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424132826 0 0) 0x7fdfe014dd30 con 0x7fdfe014b1d0 2023-10-18T18:00:29.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfd67fc700 1 -- 172.21.15.72:0/2897443114 >> v1:172.21.15.179:6789/0 conn(0x7fdfe00bf8d0 legacy=0x7fdfe01586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:29.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfd67fc700 1 -- 172.21.15.72:0/2897443114 >> v1:172.21.15.72:6789/0 conn(0x7fdfe014b1d0 legacy=0x7fdfe014e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:29.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfd67fc700 1 -- 172.21.15.72:0/2897443114 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe00629f0 con 0x7fdfe014b5b0 2023-10-18T18:00:29.437 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfd67fc700 1 --2- 172.21.15.72:0/2897443114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b5b0 0x7fdfe00bcd70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fdfc8009760 tx=0x7fdfc8006f40).ready entity=mon.2 client_cookie=d0896bec19f65c9d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:29.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfde7fc700 1 -- 172.21.15.72:0/2897443114 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8005880 con 0x7fdfe014b5b0 2023-10-18T18:00:29.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfde7fc700 1 -- 172.21.15.72:0/2897443114 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc80059e0 con 0x7fdfe014b5b0 2023-10-18T18:00:29.438 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.431+0000 7fdfde7fc700 1 -- 172.21.15.72:0/2897443114 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8005d60 con 0x7fdfe014b5b0 2023-10-18T18:00:29.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.435+0000 7fdfe58f1700 1 -- 172.21.15.72:0/2897443114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b5b0 msgr2=0x7fdfe00bcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:29.439 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.435+0000 7fdfe58f1700 1 --2- 172.21.15.72:0/2897443114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b5b0 0x7fdfe00bcd70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fdfc8009760 tx=0x7fdfc8006f40).stop 2023-10-18T18:00:29.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.435+0000 7fdfe58f1700 1 -- 172.21.15.72:0/2897443114 shutdown_connections 2023-10-18T18:00:29.440 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.435+0000 7fdfe58f1700 1 --2- 172.21.15.72:0/2897443114 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b5b0 0x7fdfe00bcd70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:29.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.435+0000 7fdfe58f1700 1 -- 172.21.15.72:0/2897443114 >> 172.21.15.72:0/2897443114 conn(0x7fdfe00699e0 msgr2=0x7fdfe0069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:29.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.435+0000 7fdfe58f1700 1 -- 172.21.15.72:0/2897443114 shutdown_connections 2023-10-18T18:00:29.441 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.435+0000 7fdfe58f1700 1 -- 172.21.15.72:0/2897443114 wait complete. 2023-10-18T18:00:29.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.435+0000 7fdfe58f1700 1 Processor -- start 2023-10-18T18:00:29.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.435+0000 7fdfe58f1700 1 -- start start 2023-10-18T18:00:29.445 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfe58f1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfe00bf8d0 0x7fdfe0163970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:29.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfe58f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b1d0 0x7fdfe0148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:29.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfe58f1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfe0163e60 0x7fdfe0149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:29.446 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfe58f1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe00b6180 con 0x7fdfe0163e60 2023-10-18T18:00:29.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfe58f1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe00b5ba0 con 0x7fdfe00bf8d0 2023-10-18T18:00:29.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfe58f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0164680 con 0x7fdfe014b1d0 2023-10-18T18:00:29.447 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfe0163e60 0x7fdfe0149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:29.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfe0163e60 0x7fdfe0149410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:44306/0 (socket says 172.21.15.72:44306) 2023-10-18T18:00:29.448 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfd67fc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b1d0 0x7fdfe0148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:29.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdeffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfe00bf8d0 0x7fdfe0163970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:29.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdffff700 1 -- 172.21.15.72:0/872980322 learned_addr learned my addr 172.21.15.72:0/872980322 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:29.449 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdffff700 1 -- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b1d0 msgr2=0x7fdfe0148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdffff700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b1d0 0x7fdfe0148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdffff700 1 -- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfe00bf8d0 msgr2=0x7fdfe0163970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:29.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdffff700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfe00bf8d0 0x7fdfe0163970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdffff700 1 -- 172.21.15.72:0/872980322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8009250 con 0x7fdfe0163e60 2023-10-18T18:00:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfdffff700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfe0163e60 0x7fdfe0149410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fdfd800c7f0 tx=0x7fdfd8007f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfd77fe700 1 -- 172.21.15.72:0/872980322 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800e4a0 con 0x7fdfe0163e60 2023-10-18T18:00:29.451 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfd77fe700 1 -- 172.21.15.72:0/872980322 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd800e600 con 0x7fdfe0163e60 2023-10-18T18:00:29.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfd77fe700 1 -- 172.21.15.72:0/872980322 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800e9a0 con 0x7fdfe0163e60 2023-10-18T18:00:29.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe00b5ea0 con 0x7fdfe0163e60 2023-10-18T18:00:29.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.439+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe0164310 con 0x7fdfe0163e60 2023-10-18T18:00:29.452 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.443+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe006c190 con 0x7fdfe0163e60 2023-10-18T18:00:29.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.443+0000 7fdfd77fe700 1 -- 172.21.15.72:0/872980322 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fdfd800ed10 con 0x7fdfe0163e60 2023-10-18T18:00:29.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.443+0000 7fdfd77fe700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdfc005a4e0 0x7fdfc005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:29.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.443+0000 7fdfd77fe700 1 -- 172.21.15.72:0/872980322 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fdfd80741e0 con 0x7fdfe0163e60 2023-10-18T18:00:29.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.443+0000 7fdfdeffd700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdfc005a4e0 0x7fdfc005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:29.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.447+0000 7fdfdeffd700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdfc005a4e0 0x7fdfc005c9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fdfd0007fd0 tx=0x7fdfd0007dd0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:29.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.451+0000 7fdfd77fe700 1 -- 172.21.15.72:0/872980322 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fdfd8047730 con 0x7fdfe0163e60 2023-10-18T18:00:29.647 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.643+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdfe0161740 con 0x7fdfe0163e60 2023-10-18T18:00:29.648 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.643+0000 7fdfd77fe700 1 -- 172.21.15.72:0/872980322 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fdfd8017020 con 0x7fdfe0163e60 2023-10-18T18:00:29.648 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:29.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdfc005a4e0 msgr2=0x7fdfc005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:29.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdfc005a4e0 0x7fdfc005c9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fdfd0007fd0 tx=0x7fdfd0007dd0).stop 2023-10-18T18:00:29.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfe0163e60 msgr2=0x7fdfe0149410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:29.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfe0163e60 0x7fdfe0149410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fdfd800c7f0 tx=0x7fdfd8007f20).stop 2023-10-18T18:00:29.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 shutdown_connections 2023-10-18T18:00:29.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fdfc005a4e0 0x7fdfc005c9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:29.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdfe0163e60 0x7fdfe0149410 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:29.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fdfe014b1d0 0x7fdfe0148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:29.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 --2- 172.21.15.72:0/872980322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfe00bf8d0 0x7fdfe0163970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:29.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 >> 172.21.15.72:0/872980322 conn(0x7fdfe00699e0 msgr2=0x7fdfe0069dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:29.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 shutdown_connections 2023-10-18T18:00:29.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:29.647+0000 7fdfe58f1700 1 -- 172.21.15.72:0/872980322 wait complete. 2023-10-18T18:00:29.673 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:30.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~119s 2023-10-18T18:00:30.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~65s 2023-10-18T18:00:30.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~19s 2023-10-18T18:00:30.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~45s 2023-10-18T18:00:30.069 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~32s 2023-10-18T18:00:30.069 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~78s 2023-10-18T18:00:32.674 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:32.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.831+0000 7f8206b69700 1 Processor -- start 2023-10-18T18:00:32.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.831+0000 7f8206b69700 1 -- start start 2023-10-18T18:00:32.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8206b69700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8200067ff0 0x7f82000683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:32.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8206b69700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8200151410 0x7f8200151820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:32.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8206b69700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8200065cd0 con 0x7f82000666b0 2023-10-18T18:00:32.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8206b69700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8200066160 con 0x7f8200151410 2023-10-18T18:00:32.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8206b69700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8200154750 con 0x7f8200067ff0 2023-10-18T18:00:32.839 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f81fffff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f82000666b0 0x7f82000bd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42818/0 (socket says 172.21.15.72:42818) 2023-10-18T18:00:32.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8204905700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8200067ff0 0x7f82000683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:32.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f81fffff700 1 -- 172.21.15.72:0/3009030881 learned_addr learned my addr 172.21.15.72:0/3009030881 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:32.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8205106700 1 --2- 172.21.15.72:0/3009030881 >> v2:172.21.15.72:3300/0 conn(0x7f8200151410 0x7f8200151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:32.840 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8205106700 1 -- 172.21.15.72:0/3009030881 >> v2:172.21.15.179:3300/0 conn(0x7f8200067ff0 msgr2=0x7f82000683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:32.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8205106700 1 --2- 172.21.15.72:0/3009030881 >> v2:172.21.15.179:3300/0 conn(0x7f8200067ff0 0x7f82000683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:32.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8205106700 1 -- 172.21.15.72:0/3009030881 >> v1:172.21.15.72:6789/0 conn(0x7f82000666b0 legacy=0x7f82000bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:32.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8205106700 1 -- 172.21.15.72:0/3009030881 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8200065100 con 0x7f8200151410 2023-10-18T18:00:32.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f8205106700 1 --2- 172.21.15.72:0/3009030881 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8200151410 0x7f8200151820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f820005e8c0 tx=0x7f81f8009780).ready entity=mon.0 client_cookie=a1e840862629ec00 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:32.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f81ff7fe700 1 -- 172.21.15.72:0/3009030881 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f800e070 con 0x7f8200151410 2023-10-18T18:00:32.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f81ff7fe700 1 -- 172.21.15.72:0/3009030881 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f800bb60 con 0x7f8200151410 2023-10-18T18:00:32.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.835+0000 7f81ff7fe700 1 -- 172.21.15.72:0/3009030881 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8012410 con 0x7f8200151410 2023-10-18T18:00:32.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 -- 172.21.15.72:0/3009030881 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8200151410 msgr2=0x7f8200151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:32.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 --2- 172.21.15.72:0/3009030881 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8200151410 0x7f8200151820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f820005e8c0 tx=0x7f81f8009780).stop 2023-10-18T18:00:32.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 -- 172.21.15.72:0/3009030881 shutdown_connections 2023-10-18T18:00:32.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 --2- 172.21.15.72:0/3009030881 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8200151410 0x7f8200151820 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:32.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 --2- 172.21.15.72:0/3009030881 >> v2:172.21.15.179:3300/0 conn(0x7f8200067ff0 0x7f82000683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:32.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 -- 172.21.15.72:0/3009030881 >> 172.21.15.72:0/3009030881 conn(0x7f82000620d0 msgr2=0x7f82000624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:32.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 -- 172.21.15.72:0/3009030881 shutdown_connections 2023-10-18T18:00:32.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 -- 172.21.15.72:0/3009030881 wait complete. 2023-10-18T18:00:32.845 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 Processor -- start 2023-10-18T18:00:32.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.839+0000 7f8206b69700 1 -- start start 2023-10-18T18:00:32.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8206b69700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82000666b0 0x7f8200153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:32.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8206b69700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8200067ff0 0x7f82001647b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:32.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8206b69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8200151410 0x7f8200164ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:32.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8206b69700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8200165190 con 0x7f82000666b0 2023-10-18T18:00:32.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8206b69700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82000b8660 con 0x7f8200067ff0 2023-10-18T18:00:32.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8206b69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82000b6f60 con 0x7f8200151410 2023-10-18T18:00:32.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f81fffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8200067ff0 0x7f82001647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:32.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8204905700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82000666b0 0x7f8200153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:32.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f81fffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8200067ff0 0x7f82001647b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:40782/0 (socket says 172.21.15.72:40782) 2023-10-18T18:00:32.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f81fffff700 1 -- 172.21.15.72:0/45421731 learned_addr learned my addr 172.21.15.72:0/45421731 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:32.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8205106700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8200151410 0x7f8200164ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:32.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8204905700 1 -- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8200151410 msgr2=0x7f8200164ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:32.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8204905700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8200151410 0x7f8200164ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:32.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8204905700 1 -- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8200067ff0 msgr2=0x7f82001647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:32.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8204905700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8200067ff0 0x7f82001647b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:32.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8204905700 1 -- 172.21.15.72:0/45421731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8009a20 con 0x7f82000666b0 2023-10-18T18:00:32.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8204905700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82000666b0 0x7f8200153910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f81f400e900 tx=0x7f81f4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:32.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f81fcff9700 1 -- 172.21.15.72:0/45421731 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4009450 con 0x7f82000666b0 2023-10-18T18:00:32.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f81fcff9700 1 -- 172.21.15.72:0/45421731 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f4012600 con 0x7f82000666b0 2023-10-18T18:00:32.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f81fcff9700 1 -- 172.21.15.72:0/45421731 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4012980 con 0x7f82000666b0 2023-10-18T18:00:32.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82000bba60 con 0x7f82000666b0 2023-10-18T18:00:32.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.843+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82000bc020 con 0x7f82000666b0 2023-10-18T18:00:32.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.847+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820006c190 con 0x7f82000666b0 2023-10-18T18:00:32.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.847+0000 7f81fcff9700 1 -- 172.21.15.72:0/45421731 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f81f4012cf0 con 0x7f82000666b0 2023-10-18T18:00:32.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.847+0000 7f81fcff9700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f81e405a540 0x7f81e405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:32.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.847+0000 7f81fcff9700 1 -- 172.21.15.72:0/45421731 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f81f4074bf0 con 0x7f82000666b0 2023-10-18T18:00:32.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.847+0000 7f81fffff700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f81e405a540 0x7f81e405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:32.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.851+0000 7f81fffff700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f81e405a540 0x7f81e405ca00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f82000c22b0 tx=0x7f81ec009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:32.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:32.855+0000 7f81fcff9700 1 -- 172.21.15.72:0/45421731 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f81f4048200 con 0x7f82000666b0 2023-10-18T18:00:33.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.051+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f82000653a0 con 0x7f82000666b0 2023-10-18T18:00:33.055 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.051+0000 7f81fcff9700 1 -- 172.21.15.72:0/45421731 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f81f404b820 con 0x7f82000666b0 2023-10-18T18:00:33.055 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:33.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f81e405a540 msgr2=0x7f81e405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:33.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f81e405a540 0x7f81e405ca00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f82000c22b0 tx=0x7f81ec009040).stop 2023-10-18T18:00:33.060 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82000666b0 msgr2=0x7f8200153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:33.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82000666b0 0x7f8200153910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f81f400e900 tx=0x7f81f4009060).stop 2023-10-18T18:00:33.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 shutdown_connections 2023-10-18T18:00:33.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f81e405a540 0x7f81e405ca00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:33.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8200151410 0x7f8200164ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:33.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8200067ff0 0x7f82001647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:33.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 --2- 172.21.15.72:0/45421731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82000666b0 0x7f8200153910 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:33.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 >> 172.21.15.72:0/45421731 conn(0x7f82000620d0 msgr2=0x7f8200154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:33.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 shutdown_connections 2023-10-18T18:00:33.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:33.055+0000 7f8206b69700 1 -- 172.21.15.72:0/45421731 wait complete. 2023-10-18T18:00:33.081 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:36.082 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:36.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.231+0000 7f2ce6a39700 1 Processor -- start 2023-10-18T18:00:36.237 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.235+0000 7f2ce6a39700 1 -- start start 2023-10-18T18:00:36.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.235+0000 7f2ce6a39700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2ce00630e0 0x7f2ce0065700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:36.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.235+0000 7f2ce6a39700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce0154b00 con 0x7f2ce01513e0 2023-10-18T18:00:36.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.235+0000 7f2ce6a39700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce006a750 con 0x7f2ce01517c0 2023-10-18T18:00:36.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.235+0000 7f2ce6a39700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce0069380 con 0x7f2ce00630e0 2023-10-18T18:00:36.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce5a37700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f2ce01513e0 0x7f2ce0067d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42830/0 (socket says 172.21.15.72:42830) 2023-10-18T18:00:36.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce5a37700 1 -- 172.21.15.72:0/4068390281 learned_addr learned my addr 172.21.15.72:0/4068390281 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:36.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6238700 1 --2- 172.21.15.72:0/4068390281 >> v2:172.21.15.72:3300/0 conn(0x7f2ce00630e0 0x7f2ce0065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:36.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce4a35700 1 -- 172.21.15.72:0/4068390281 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945877295 0 0) 0x7f2ce006a750 con 0x7f2ce01517c0 2023-10-18T18:00:36.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce4a35700 1 -- 172.21.15.72:0/4068390281 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ccc0033f0 con 0x7f2ce01517c0 2023-10-18T18:00:36.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce4a35700 1 -- 172.21.15.72:0/4068390281 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498115680 0 0) 0x7f2ce0154b00 con 0x7f2ce01513e0 2023-10-18T18:00:36.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce4a35700 1 -- 172.21.15.72:0/4068390281 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce006a750 con 0x7f2ce01513e0 2023-10-18T18:00:36.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6238700 1 -- 172.21.15.72:0/4068390281 >> v1:172.21.15.179:6789/0 conn(0x7f2ce01517c0 legacy=0x7f2ce006a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:36.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6238700 1 -- 172.21.15.72:0/4068390281 >> v1:172.21.15.72:6789/0 conn(0x7f2ce01513e0 legacy=0x7f2ce0067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:36.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6238700 1 -- 172.21.15.72:0/4068390281 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0154f90 con 0x7f2ce00630e0 2023-10-18T18:00:36.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6238700 1 --2- 172.21.15.72:0/4068390281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 0x7f2ce0065700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f2cd8009760 tx=0x7f2cd8009060).ready entity=mon.0 client_cookie=33ea483ee4b1d7c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:36.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce4a35700 1 -- 172.21.15.72:0/4068390281 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800d4a0 con 0x7f2ce00630e0 2023-10-18T18:00:36.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce4a35700 1 -- 172.21.15.72:0/4068390281 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd800d600 con 0x7f2ce00630e0 2023-10-18T18:00:36.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce4a35700 1 -- 172.21.15.72:0/4068390281 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800d980 con 0x7f2ce00630e0 2023-10-18T18:00:36.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6a39700 1 -- 172.21.15.72:0/4068390281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 msgr2=0x7f2ce0065700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:36.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6a39700 1 --2- 172.21.15.72:0/4068390281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 0x7f2ce0065700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f2cd8009760 tx=0x7f2cd8009060).stop 2023-10-18T18:00:36.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6a39700 1 -- 172.21.15.72:0/4068390281 shutdown_connections 2023-10-18T18:00:36.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6a39700 1 --2- 172.21.15.72:0/4068390281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 0x7f2ce0065700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:36.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6a39700 1 -- 172.21.15.72:0/4068390281 >> 172.21.15.72:0/4068390281 conn(0x7f2ce0062ba0 msgr2=0x7f2ce00c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:36.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6a39700 1 -- 172.21.15.72:0/4068390281 shutdown_connections 2023-10-18T18:00:36.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.239+0000 7f2ce6a39700 1 -- 172.21.15.72:0/4068390281 wait complete. 2023-10-18T18:00:36.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce6a39700 1 Processor -- start 2023-10-18T18:00:36.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce6a39700 1 -- start start 2023-10-18T18:00:36.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce6a39700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 0x7f2ce014cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:36.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce6a39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce01513e0 0x7f2ce0164d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:36.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce6a39700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ce01517c0 0x7f2ce00bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:36.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce6a39700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0165220 con 0x7f2ce00630e0 2023-10-18T18:00:36.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce6a39700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce01462a0 con 0x7f2ce01513e0 2023-10-18T18:00:36.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce6a39700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce0153400 con 0x7f2ce01517c0 2023-10-18T18:00:36.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce5a37700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 0x7f2ce014cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:36.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce5a37700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 0x7f2ce014cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54918/0 (socket says 172.21.15.72:54918) 2023-10-18T18:00:36.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce5a37700 1 -- 172.21.15.72:0/3915573093 learned_addr learned my addr 172.21.15.72:0/3915573093 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:36.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce6238700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ce01517c0 0x7f2ce00bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:36.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce5236700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce01513e0 0x7f2ce0164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:36.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.243+0000 7f2ce5a37700 1 -- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ce01517c0 msgr2=0x7f2ce00bbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:36.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2ce5a37700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ce01517c0 0x7f2ce00bbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:36.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2ce5a37700 1 -- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce01513e0 msgr2=0x7f2ce0164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:36.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2ce5a37700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce01513e0 0x7f2ce0164d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:36.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2ce5a37700 1 -- 172.21.15.72:0/3915573093 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd80092a0 con 0x7f2ce00630e0 2023-10-18T18:00:36.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2ce5a37700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 0x7f2ce014cf80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f2cdc00c850 tx=0x7f2cdc001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:36.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2cd5ffb700 1 -- 172.21.15.72:0/3915573093 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc005d30 con 0x7f2ce00630e0 2023-10-18T18:00:36.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2cd5ffb700 1 -- 172.21.15.72:0/3915573093 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc005e90 con 0x7f2ce00630e0 2023-10-18T18:00:36.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce0152970 con 0x7f2ce00630e0 2023-10-18T18:00:36.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2cd5ffb700 1 -- 172.21.15.72:0/3915573093 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc007320 con 0x7f2ce00630e0 2023-10-18T18:00:36.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce014c8d0 con 0x7f2ce00630e0 2023-10-18T18:00:36.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce0046f00 con 0x7f2ce00630e0 2023-10-18T18:00:36.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.247+0000 7f2cd5ffb700 1 -- 172.21.15.72:0/3915573093 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2cdc007710 con 0x7f2ce00630e0 2023-10-18T18:00:36.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.251+0000 7f2cd5ffb700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2cc405a4e0 0x7f2cc405c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:36.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.251+0000 7f2cd5ffb700 1 -- 172.21.15.72:0/3915573093 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f2cdc074a70 con 0x7f2ce00630e0 2023-10-18T18:00:36.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.251+0000 7f2ce5236700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2cc405a4e0 0x7f2cc405c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:36.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.251+0000 7f2ce5236700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2cc405a4e0 0x7f2cc405c9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f2ce014bad0 tx=0x7f2cd0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:36.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.255+0000 7f2cd5ffb700 1 -- 172.21.15.72:0/3915573093 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f2cdc047fc0 con 0x7f2ce00630e0 2023-10-18T18:00:36.380 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~125s 2023-10-18T18:00:36.380 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~71s 2023-10-18T18:00:36.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~25s 2023-10-18T18:00:36.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~51s 2023-10-18T18:00:36.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~38s 2023-10-18T18:00:36.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~84s 2023-10-18T18:00:36.453 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.451+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2ce00c1480 con 0x7f2ce00630e0 2023-10-18T18:00:36.454 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.451+0000 7f2cd5ffb700 1 -- 172.21.15.72:0/3915573093 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f2cdc04b5e0 con 0x7f2ce00630e0 2023-10-18T18:00:36.454 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:36.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2cc405a4e0 msgr2=0x7f2cc405c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:36.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2cc405a4e0 0x7f2cc405c9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f2ce014bad0 tx=0x7f2cd0009040).stop 2023-10-18T18:00:36.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 msgr2=0x7f2ce014cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:36.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 0x7f2ce014cf80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f2cdc00c850 tx=0x7f2cdc001650).stop 2023-10-18T18:00:36.459 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 shutdown_connections 2023-10-18T18:00:36.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f2cc405a4e0 0x7f2cc405c9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:36.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2ce01517c0 0x7f2ce00bbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:36.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ce01513e0 0x7f2ce0164d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:36.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 --2- 172.21.15.72:0/3915573093 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ce00630e0 0x7f2ce014cf80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:36.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 >> 172.21.15.72:0/3915573093 conn(0x7f2ce0062ba0 msgr2=0x7f2ce0159990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:36.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 shutdown_connections 2023-10-18T18:00:36.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:36.455+0000 7f2ce6a39700 1 -- 172.21.15.72:0/3915573093 wait complete. 2023-10-18T18:00:36.478 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:39.479 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:39.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.647+0000 7f6df31f9700 1 Processor -- start 2023-10-18T18:00:39.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.647+0000 7f6df31f9700 1 -- start start 2023-10-18T18:00:39.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.651+0000 7f6df31f9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6dec0658a0 0x7f6dec065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:39.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.651+0000 7f6df31f9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6dec0c2bf0 0x7f6dec0630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:39.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.651+0000 7f6df31f9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dec0be2e0 con 0x7f6dec0635d0 2023-10-18T18:00:39.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.651+0000 7f6df31f9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec0bda80 con 0x7f6dec0658a0 2023-10-18T18:00:39.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.651+0000 7f6df31f9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec06a7c0 con 0x7f6dec0c2bf0 2023-10-18T18:00:39.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.651+0000 7f6df21f7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6dec0658a0 0x7f6dec065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:39.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.651+0000 7f6df21f7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6dec0658a0 0x7f6dec065c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54922/0 (socket says 172.21.15.72:54922) 2023-10-18T18:00:39.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.651+0000 7f6df21f7700 1 -- 172.21.15.72:0/1426936447 learned_addr learned my addr 172.21.15.72:0/1426936447 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:39.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.651+0000 7f6df19f6700 1 --2- 172.21.15.72:0/1426936447 >> v2:172.21.15.179:3300/0 conn(0x7f6dec0c2bf0 0x7f6dec0630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:39.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df21f7700 1 -- 172.21.15.72:0/1426936447 >> v2:172.21.15.179:3300/0 conn(0x7f6dec0c2bf0 msgr2=0x7f6dec0630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:39.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df21f7700 1 --2- 172.21.15.72:0/1426936447 >> v2:172.21.15.179:3300/0 conn(0x7f6dec0c2bf0 0x7f6dec0630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:39.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df21f7700 1 -- 172.21.15.72:0/1426936447 >> v1:172.21.15.72:6789/0 conn(0x7f6dec0635d0 legacy=0x7f6dec151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:39.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df21f7700 1 -- 172.21.15.72:0/1426936447 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0be770 con 0x7f6dec0658a0 2023-10-18T18:00:39.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df11f5700 1 -- 172.21.15.72:0/1426936447 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848438583 0 0) 0x7f6dec0be2e0 con 0x7f6dec0635d0 2023-10-18T18:00:39.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df21f7700 1 --2- 172.21.15.72:0/1426936447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0658a0 0x7f6dec065c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6de8009f20 tx=0x7f6de8009840).ready entity=mon.0 client_cookie=7768399f599f0955 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:39.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df11f5700 1 -- 172.21.15.72:0/1426936447 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800e070 con 0x7f6dec0658a0 2023-10-18T18:00:39.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df11f5700 1 -- 172.21.15.72:0/1426936447 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de800b990 con 0x7f6dec0658a0 2023-10-18T18:00:39.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df11f5700 1 -- 172.21.15.72:0/1426936447 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800bd10 con 0x7f6dec0658a0 2023-10-18T18:00:39.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df31f9700 1 -- 172.21.15.72:0/1426936447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0658a0 msgr2=0x7f6dec065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:39.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df31f9700 1 --2- 172.21.15.72:0/1426936447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0658a0 0x7f6dec065c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6de8009f20 tx=0x7f6de8009840).stop 2023-10-18T18:00:39.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df31f9700 1 -- 172.21.15.72:0/1426936447 shutdown_connections 2023-10-18T18:00:39.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df31f9700 1 --2- 172.21.15.72:0/1426936447 >> v2:172.21.15.179:3300/0 conn(0x7f6dec0c2bf0 0x7f6dec0630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:39.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df31f9700 1 --2- 172.21.15.72:0/1426936447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0658a0 0x7f6dec065c80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:39.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df31f9700 1 -- 172.21.15.72:0/1426936447 >> 172.21.15.72:0/1426936447 conn(0x7f6dec062ba0 msgr2=0x7f6dec0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:39.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df31f9700 1 -- 172.21.15.72:0/1426936447 shutdown_connections 2023-10-18T18:00:39.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.655+0000 7f6df31f9700 1 -- 172.21.15.72:0/1426936447 wait complete. 2023-10-18T18:00:39.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df31f9700 1 Processor -- start 2023-10-18T18:00:39.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df31f9700 1 -- start start 2023-10-18T18:00:39.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df31f9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0635d0 0x7f6dec15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:39.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df31f9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dec0658a0 0x7f6dec0c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:39.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df31f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dec0c2bf0 0x7f6dec150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:39.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df31f9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0bfd30 con 0x7f6dec0635d0 2023-10-18T18:00:39.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df31f9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0bff10 con 0x7f6dec0c2bf0 2023-10-18T18:00:39.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df31f9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0c01f0 con 0x7f6dec0658a0 2023-10-18T18:00:39.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df21f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0635d0 0x7f6dec15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:39.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df29f8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dec0c2bf0 0x7f6dec150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:39.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df21f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0635d0 0x7f6dec15b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54926/0 (socket says 172.21.15.72:54926) 2023-10-18T18:00:39.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df19f6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dec0658a0 0x7f6dec0c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:39.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df21f7700 1 -- 172.21.15.72:0/896549075 learned_addr learned my addr 172.21.15.72:0/896549075 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:39.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df21f7700 1 -- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dec0658a0 msgr2=0x7f6dec0c0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:39.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df21f7700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dec0658a0 0x7f6dec0c0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:39.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df21f7700 1 -- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dec0c2bf0 msgr2=0x7f6dec150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:39.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df21f7700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dec0c2bf0 0x7f6dec150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:39.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df21f7700 1 -- 172.21.15.72:0/896549075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de8009ae0 con 0x7f6dec0635d0 2023-10-18T18:00:39.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.659+0000 7f6df29f8700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dec0c2bf0 0x7f6dec150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:00:39.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.663+0000 7f6df21f7700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0635d0 0x7f6dec15b9e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f6de800b610 tx=0x7f6de8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:39.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.663+0000 7f6de27fc700 1 -- 172.21.15.72:0/896549075 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800bec0 con 0x7f6dec0635d0 2023-10-18T18:00:39.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.663+0000 7f6de27fc700 1 -- 172.21.15.72:0/896549075 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de800bb00 con 0x7f6dec0635d0 2023-10-18T18:00:39.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.663+0000 7f6de27fc700 1 -- 172.21.15.72:0/896549075 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de801d620 con 0x7f6dec0635d0 2023-10-18T18:00:39.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.663+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec0c04d0 con 0x7f6dec0635d0 2023-10-18T18:00:39.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.663+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec0c0a90 con 0x7f6dec0635d0 2023-10-18T18:00:39.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.663+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec046f00 con 0x7f6dec0635d0 2023-10-18T18:00:39.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.663+0000 7f6de27fc700 1 -- 172.21.15.72:0/896549075 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6de801d9d0 con 0x7f6dec0635d0 2023-10-18T18:00:39.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.667+0000 7f6de27fc700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6dd005a510 0x7f6dd005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:39.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.667+0000 7f6de27fc700 1 -- 172.21.15.72:0/896549075 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f6de8076370 con 0x7f6dec0635d0 2023-10-18T18:00:39.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.667+0000 7f6df19f6700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6dd005a510 0x7f6dd005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:39.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.667+0000 7f6df19f6700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6dd005a510 0x7f6dd005c9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f6dec0b5be0 tx=0x7f6ddc009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:39.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.675+0000 7f6de27fc700 1 -- 172.21.15.72:0/896549075 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f6de80498c0 con 0x7f6dec0635d0 2023-10-18T18:00:39.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.867+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6dec0a8050 con 0x7f6dec0635d0 2023-10-18T18:00:39.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.867+0000 7f6de27fc700 1 -- 172.21.15.72:0/896549075 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f6de801c370 con 0x7f6dec0635d0 2023-10-18T18:00:39.872 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:39.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6dd005a510 msgr2=0x7f6dd005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:39.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6dd005a510 0x7f6dd005c9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f6dec0b5be0 tx=0x7f6ddc009060).stop 2023-10-18T18:00:39.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0635d0 msgr2=0x7f6dec15b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:39.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0635d0 0x7f6dec15b9e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f6de800b610 tx=0x7f6de8009840).stop 2023-10-18T18:00:39.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 shutdown_connections 2023-10-18T18:00:39.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6dd005a510 0x7f6dd005c9d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:39.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dec0c2bf0 0x7f6dec150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:39.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6dec0658a0 0x7f6dec0c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:39.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 --2- 172.21.15.72:0/896549075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6dec0635d0 0x7f6dec15b9e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:39.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 >> 172.21.15.72:0/896549075 conn(0x7f6dec062ba0 msgr2=0x7f6dec153e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:39.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 shutdown_connections 2023-10-18T18:00:39.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:39.871+0000 7f6df31f9700 1 -- 172.21.15.72:0/896549075 wait complete. 2023-10-18T18:00:39.893 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:42.691 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~132s 2023-10-18T18:00:42.692 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~77s 2023-10-18T18:00:42.692 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~32s 2023-10-18T18:00:42.692 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~57s 2023-10-18T18:00:42.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~44s 2023-10-18T18:00:42.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~91s 2023-10-18T18:00:42.894 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:43.058 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.055+0000 7f9440db9700 1 Processor -- start 2023-10-18T18:00:43.059 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.055+0000 7f9440db9700 1 -- start start 2023-10-18T18:00:43.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f9440db9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c1513e0 0x7f943c067d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:43.061 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f9440db9700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f943c0630e0 0x7f943c065700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:43.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f9440db9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c154b00 con 0x7f943c1517c0 2023-10-18T18:00:43.062 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f9440db9700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c069380 con 0x7f943c0630e0 2023-10-18T18:00:43.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f9440db9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c06a750 con 0x7f943c1513e0 2023-10-18T18:00:43.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943b7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c1513e0 0x7f943c067d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:43.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f9432ffd700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f943c1517c0 0x7f943c06a040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:55866/0 (socket says 172.21.15.72:55866) 2023-10-18T18:00:43.063 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f9432ffd700 1 -- 172.21.15.72:0/2513671735 learned_addr learned my addr 172.21.15.72:0/2513671735 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:43.064 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943bfff700 1 --2- 172.21.15.72:0/2513671735 >> v2:172.21.15.179:3300/0 conn(0x7f943c0630e0 0x7f943c065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:43.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943affd700 1 -- 172.21.15.72:0/2513671735 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490663654 0 0) 0x7f943c154b00 con 0x7f943c1517c0 2023-10-18T18:00:43.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943affd700 1 -- 172.21.15.72:0/2513671735 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94280033f0 con 0x7f943c1517c0 2023-10-18T18:00:43.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943b7fe700 1 -- 172.21.15.72:0/2513671735 >> v2:172.21.15.179:3300/0 conn(0x7f943c0630e0 msgr2=0x7f943c065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:43.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943b7fe700 1 --2- 172.21.15.72:0/2513671735 >> v2:172.21.15.179:3300/0 conn(0x7f943c0630e0 0x7f943c065700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:43.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943b7fe700 1 -- 172.21.15.72:0/2513671735 >> v1:172.21.15.179:6789/0 conn(0x7f943c1517c0 legacy=0x7f943c06a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:43.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943b7fe700 1 -- 172.21.15.72:0/2513671735 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c154f90 con 0x7f943c1513e0 2023-10-18T18:00:43.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943b7fe700 1 --2- 172.21.15.72:0/2513671735 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c1513e0 0x7f943c067d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f942c009e60 tx=0x7f942c009780).ready entity=mon.2 client_cookie=9d52e9cfe79f730b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:43.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943affd700 1 -- 172.21.15.72:0/2513671735 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00e070 con 0x7f943c1513e0 2023-10-18T18:00:43.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943affd700 1 -- 172.21.15.72:0/2513671735 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c00bb40 con 0x7f943c1513e0 2023-10-18T18:00:43.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.059+0000 7f943affd700 1 -- 172.21.15.72:0/2513671735 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c012410 con 0x7f943c1513e0 2023-10-18T18:00:43.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 -- 172.21.15.72:0/2513671735 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c1513e0 msgr2=0x7f943c067d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:43.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 --2- 172.21.15.72:0/2513671735 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c1513e0 0x7f943c067d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f942c009e60 tx=0x7f942c009780).stop 2023-10-18T18:00:43.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 -- 172.21.15.72:0/2513671735 shutdown_connections 2023-10-18T18:00:43.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 --2- 172.21.15.72:0/2513671735 >> v2:172.21.15.179:3300/0 conn(0x7f943c0630e0 0x7f943c065700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:43.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 --2- 172.21.15.72:0/2513671735 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c1513e0 0x7f943c067d30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:43.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 -- 172.21.15.72:0/2513671735 >> 172.21.15.72:0/2513671735 conn(0x7f943c062ba0 msgr2=0x7f943c0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:43.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 -- 172.21.15.72:0/2513671735 shutdown_connections 2023-10-18T18:00:43.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 -- 172.21.15.72:0/2513671735 wait complete. 2023-10-18T18:00:43.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 Processor -- start 2023-10-18T18:00:43.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 -- start start 2023-10-18T18:00:43.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c0630e0 0x7f943c0a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:43.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f943c1513e0 0x7f943c158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:43.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f943c1517c0 0x7f943c14e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:43.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c165c80 con 0x7f943c1517c0 2023-10-18T18:00:43.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c165750 con 0x7f943c1513e0 2023-10-18T18:00:43.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.063+0000 7f9440db9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c068bd0 con 0x7f943c0630e0 2023-10-18T18:00:43.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f943c1517c0 0x7f943c14e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:43.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f943c1517c0 0x7f943c14e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57070/0 (socket says 172.21.15.72:57070) 2023-10-18T18:00:43.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943bfff700 1 -- 172.21.15.72:0/1371702333 learned_addr learned my addr 172.21.15.72:0/1371702333 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:43.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f9432ffd700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f943c1513e0 0x7f943c158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:43.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943b7fe700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c0630e0 0x7f943c0a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:43.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943bfff700 1 -- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c0630e0 msgr2=0x7f943c0a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:43.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943bfff700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c0630e0 0x7f943c0a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:43.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943bfff700 1 -- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f943c1513e0 msgr2=0x7f943c158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:43.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943bfff700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f943c1513e0 0x7f943c158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:43.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943bfff700 1 -- 172.21.15.72:0/1371702333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009a20 con 0x7f943c1517c0 2023-10-18T18:00:43.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f943bfff700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f943c1517c0 0x7f943c14e740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f943400e900 tx=0x7f9434009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:43.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f9433fff700 1 -- 172.21.15.72:0/1371702333 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434009530 con 0x7f943c1517c0 2023-10-18T18:00:43.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f9433fff700 1 -- 172.21.15.72:0/1371702333 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434012600 con 0x7f943c1517c0 2023-10-18T18:00:43.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f9433fff700 1 -- 172.21.15.72:0/1371702333 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434012980 con 0x7f943c1517c0 2023-10-18T18:00:43.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c1659a0 con 0x7f943c1517c0 2023-10-18T18:00:43.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.067+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c1579a0 con 0x7f943c1517c0 2023-10-18T18:00:43.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.071+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c046f00 con 0x7f943c1517c0 2023-10-18T18:00:43.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.071+0000 7f9433fff700 1 -- 172.21.15.72:0/1371702333 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9434012d30 con 0x7f943c1517c0 2023-10-18T18:00:43.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.071+0000 7f9433fff700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f941c05a510 0x7f941c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:43.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.071+0000 7f9433fff700 1 -- 172.21.15.72:0/1371702333 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f9434074d90 con 0x7f943c1517c0 2023-10-18T18:00:43.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.071+0000 7f943b7fe700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f941c05a510 0x7f941c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:43.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.071+0000 7f943b7fe700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f941c05a510 0x7f941c05c9d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f942c009fd0 tx=0x7f942c009780).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:43.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.079+0000 7f9433fff700 1 -- 172.21.15.72:0/1371702333 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f94340482e0 con 0x7f943c1517c0 2023-10-18T18:00:43.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.271+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f943c068570 con 0x7f943c1517c0 2023-10-18T18:00:43.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.275+0000 7f9433fff700 1 -- 172.21.15.72:0/1371702333 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f943404b900 con 0x7f943c1517c0 2023-10-18T18:00:43.277 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:43.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f941c05a510 msgr2=0x7f941c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:43.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f941c05a510 0x7f941c05c9d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f942c009fd0 tx=0x7f942c009780).stop 2023-10-18T18:00:43.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f943c1517c0 msgr2=0x7f943c14e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:43.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f943c1517c0 0x7f943c14e740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f943400e900 tx=0x7f9434009060).stop 2023-10-18T18:00:43.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 shutdown_connections 2023-10-18T18:00:43.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f941c05a510 0x7f941c05c9d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:43.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f943c1517c0 0x7f943c14e740 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:43.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f943c1513e0 0x7f943c158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:43.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 --2- 172.21.15.72:0/1371702333 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f943c0630e0 0x7f943c0a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:43.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 >> 172.21.15.72:0/1371702333 conn(0x7f943c062ba0 msgr2=0x7f943c0656e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:43.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 shutdown_connections 2023-10-18T18:00:43.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:43.279+0000 7f9440db9700 1 -- 172.21.15.72:0/1371702333 wait complete. 2023-10-18T18:00:43.302 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:46.304 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:46.462 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.459+0000 7f800e279700 1 Processor -- start 2023-10-18T18:00:46.463 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.459+0000 7f800e279700 1 -- start start 2023-10-18T18:00:46.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.459+0000 7f800e279700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 0x7f8008065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:46.464 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.459+0000 7f800e279700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f80080bd1c0 0x7f800814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:46.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.459+0000 7f800e279700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080be2e0 con 0x7f80080c2bf0 2023-10-18T18:00:46.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800e279700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80080bda80 con 0x7f80080bd1c0 2023-10-18T18:00:46.465 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800e279700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800806a7c0 con 0x7f80080658a0 2023-10-18T18:00:46.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800da78700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f80080bd1c0 0x7f800814ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:46.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800ca76700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f80080c2bf0 0x7f80080630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:55870/0 (socket says 172.21.15.72:55870) 2023-10-18T18:00:46.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800ca76700 1 -- 172.21.15.72:0/585063627 learned_addr learned my addr 172.21.15.72:0/585063627 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:46.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800d277700 1 --2- 172.21.15.72:0/585063627 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 0x7f8008065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:46.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f7ffffff700 1 -- 172.21.15.72:0/585063627 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438881709 0 0) 0x7f80080be2e0 con 0x7f80080c2bf0 2023-10-18T18:00:46.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f7ffffff700 1 -- 172.21.15.72:0/585063627 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff40033f0 con 0x7f80080c2bf0 2023-10-18T18:00:46.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800d277700 1 -- 172.21.15.72:0/585063627 >> v2:172.21.15.72:3300/0 conn(0x7f80080bd1c0 msgr2=0x7f800814ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:46.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800d277700 1 --2- 172.21.15.72:0/585063627 >> v2:172.21.15.72:3300/0 conn(0x7f80080bd1c0 0x7f800814ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:46.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800d277700 1 -- 172.21.15.72:0/585063627 >> v1:172.21.15.179:6789/0 conn(0x7f80080c2bf0 legacy=0x7f80080630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:46.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800d277700 1 -- 172.21.15.72:0/585063627 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080be770 con 0x7f80080658a0 2023-10-18T18:00:46.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800da78700 1 --2- 172.21.15.72:0/585063627 >> v2:172.21.15.72:3300/0 conn(0x7f80080bd1c0 0x7f800814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:00:46.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800d277700 1 --2- 172.21.15.72:0/585063627 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 0x7f8008065c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8004009760 tx=0x7f8004009060).ready entity=mon.2 client_cookie=e2d9af03777f1c57 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:46.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f7ffffff700 1 -- 172.21.15.72:0/585063627 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400d4a0 con 0x7f80080658a0 2023-10-18T18:00:46.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f7ffffff700 1 -- 172.21.15.72:0/585063627 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800400d600 con 0x7f80080658a0 2023-10-18T18:00:46.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f7ffffff700 1 -- 172.21.15.72:0/585063627 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400d980 con 0x7f80080658a0 2023-10-18T18:00:46.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800e279700 1 -- 172.21.15.72:0/585063627 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 msgr2=0x7f8008065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:46.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800e279700 1 --2- 172.21.15.72:0/585063627 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 0x7f8008065c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8004009760 tx=0x7f8004009060).stop 2023-10-18T18:00:46.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800e279700 1 -- 172.21.15.72:0/585063627 shutdown_connections 2023-10-18T18:00:46.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800e279700 1 --2- 172.21.15.72:0/585063627 >> v2:172.21.15.72:3300/0 conn(0x7f80080bd1c0 0x7f800814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:46.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800e279700 1 --2- 172.21.15.72:0/585063627 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 0x7f8008065c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:46.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.463+0000 7f800e279700 1 -- 172.21.15.72:0/585063627 >> 172.21.15.72:0/585063627 conn(0x7f8008062ba0 msgr2=0x7f80080c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:46.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.467+0000 7f800e279700 1 -- 172.21.15.72:0/585063627 shutdown_connections 2023-10-18T18:00:46.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.467+0000 7f800e279700 1 -- 172.21.15.72:0/585063627 wait complete. 2023-10-18T18:00:46.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.467+0000 7f800e279700 1 Processor -- start 2023-10-18T18:00:46.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.467+0000 7f800e279700 1 -- start start 2023-10-18T18:00:46.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800e279700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 0x7f800814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:46.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800e279700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80080bd1c0 0x7f800814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:46.477 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800e279700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80080c2bf0 0x7f80081522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:46.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800e279700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800814ad30 con 0x7f80080bd1c0 2023-10-18T18:00:46.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800e279700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800814a120 con 0x7f80080c2bf0 2023-10-18T18:00:46.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800e279700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800814a400 con 0x7f80080658a0 2023-10-18T18:00:46.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800ca76700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80080bd1c0 0x7f800814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:46.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800ca76700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80080bd1c0 0x7f800814c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57088/0 (socket says 172.21.15.72:57088) 2023-10-18T18:00:46.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800ca76700 1 -- 172.21.15.72:0/1874149991 learned_addr learned my addr 172.21.15.72:0/1874149991 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:46.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800d277700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 0x7f800814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:46.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800da78700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80080c2bf0 0x7f80081522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:46.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800ca76700 1 -- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 msgr2=0x7f800814b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:46.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800ca76700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 0x7f800814b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:46.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800ca76700 1 -- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80080c2bf0 msgr2=0x7f80081522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:46.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800ca76700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80080c2bf0 0x7f80081522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:46.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800ca76700 1 -- 172.21.15.72:0/1874149991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000009a20 con 0x7f80080bd1c0 2023-10-18T18:00:46.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800da78700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80080c2bf0 0x7f80081522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:00:46.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800ca76700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80080bd1c0 0x7f800814c4b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f7ff800c420 tx=0x7f7ff8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:46.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f7ffd7fa700 1 -- 172.21.15.72:0/1874149991 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800ce80 con 0x7f80080bd1c0 2023-10-18T18:00:46.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f7ffd7fa700 1 -- 172.21.15.72:0/1874149991 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff80072e0 con 0x7f80080bd1c0 2023-10-18T18:00:46.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80040092a0 con 0x7f80080bd1c0 2023-10-18T18:00:46.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f7ffd7fa700 1 -- 172.21.15.72:0/1874149991 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8007660 con 0x7f80080bd1c0 2023-10-18T18:00:46.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.471+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800814a9c0 con 0x7f80080bd1c0 2023-10-18T18:00:46.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.475+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8008046f00 con 0x7f80080bd1c0 2023-10-18T18:00:46.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.475+0000 7f7ffd7fa700 1 -- 172.21.15.72:0/1874149991 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f7ff80059b0 con 0x7f80080bd1c0 2023-10-18T18:00:46.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.475+0000 7f7ffd7fa700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7fec05a510 0x7f7fec05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:46.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.475+0000 7f7ffd7fa700 1 -- 172.21.15.72:0/1874149991 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f7ff8074120 con 0x7f80080bd1c0 2023-10-18T18:00:46.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.475+0000 7f800d277700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7fec05a510 0x7f7fec05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:46.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.479+0000 7f800d277700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7fec05a510 0x7f7fec05c9d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f800400d260 tx=0x7f8004016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:46.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.483+0000 7f7ffd7fa700 1 -- 172.21.15.72:0/1874149991 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f7ff8047670 con 0x7f80080bd1c0 2023-10-18T18:00:46.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.679+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f80080a8050 con 0x7f80080bd1c0 2023-10-18T18:00:46.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.679+0000 7f7ffd7fa700 1 -- 172.21.15.72:0/1874149991 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f7ff8018070 con 0x7f80080bd1c0 2023-10-18T18:00:46.682 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:46.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7fec05a510 msgr2=0x7f7fec05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:46.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7fec05a510 0x7f7fec05c9d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f800400d260 tx=0x7f8004016000).stop 2023-10-18T18:00:46.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80080bd1c0 msgr2=0x7f800814c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:46.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80080bd1c0 0x7f800814c4b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f7ff800c420 tx=0x7f7ff8007c10).stop 2023-10-18T18:00:46.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 shutdown_connections 2023-10-18T18:00:46.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f7fec05a510 0x7f7fec05c9d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:46.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80080c2bf0 0x7f80081522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:46.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80080bd1c0 0x7f800814c4b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:46.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 --2- 172.21.15.72:0/1874149991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f80080658a0 0x7f800814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:46.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 >> 172.21.15.72:0/1874149991 conn(0x7f8008062ba0 msgr2=0x7f8008069730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:46.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 shutdown_connections 2023-10-18T18:00:46.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:46.683+0000 7f800e279700 1 -- 172.21.15.72:0/1874149991 wait complete. 2023-10-18T18:00:46.706 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:49.002 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~138s 2023-10-18T18:00:49.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~84s 2023-10-18T18:00:49.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~38s 2023-10-18T18:00:49.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~64s 2023-10-18T18:00:49.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~51s 2023-10-18T18:00:49.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~97s 2023-10-18T18:00:49.707 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.859+0000 7f59753a9700 1 Processor -- start 2023-10-18T18:00:49.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f59753a9700 1 -- start start 2023-10-18T18:00:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f59753a9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f597014d080 0x7f5970062920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f59753a9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f5970062e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:49.866 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f59753a9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597014fea0 con 0x7f597005e860 2023-10-18T18:00:49.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f59753a9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597014fcb0 con 0x7f597014d080 2023-10-18T18:00:49.867 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f59753a9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59700674e0 con 0x7f59700c5210 2023-10-18T18:00:49.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596effd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f597014d080 0x7f5970062920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:49.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f59667fc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f5970062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:49.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596effd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f597014d080 0x7f5970062920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60628/0 (socket says 172.21.15.72:60628) 2023-10-18T18:00:49.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596effd700 1 -- 172.21.15.72:0/3599418982 learned_addr learned my addr 172.21.15.72:0/3599418982 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596effd700 1 -- 172.21.15.72:0/3599418982 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 msgr2=0x7f5970062e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596effd700 1 --2- 172.21.15.72:0/3599418982 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f5970062e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596effd700 1 -- 172.21.15.72:0/3599418982 >> v1:172.21.15.179:6789/0 conn(0x7f597005e860 legacy=0x7f5970069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596effd700 1 -- 172.21.15.72:0/3599418982 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970150140 con 0x7f597014d080 2023-10-18T18:00:49.869 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596e7fc700 1 -- 172.21.15.72:0/3599418982 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514224681 0 0) 0x7f597014fea0 con 0x7f597005e860 2023-10-18T18:00:49.870 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596effd700 1 --2- 172.21.15.72:0/3599418982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597014d080 0x7f5970062920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5960006fd0 tx=0x7f59600090f0).ready entity=mon.0 client_cookie=ec266230e75b3988 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.863+0000 7f596e7fc700 1 -- 172.21.15.72:0/3599418982 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596000b610 con 0x7f597014d080 2023-10-18T18:00:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f596e7fc700 1 -- 172.21.15.72:0/3599418982 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596000b770 con 0x7f597014d080 2023-10-18T18:00:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f596e7fc700 1 -- 172.21.15.72:0/3599418982 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596000baf0 con 0x7f597014d080 2023-10-18T18:00:49.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f59753a9700 1 -- 172.21.15.72:0/3599418982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597014d080 msgr2=0x7f5970062920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:49.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f59753a9700 1 --2- 172.21.15.72:0/3599418982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597014d080 0x7f5970062920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5960006fd0 tx=0x7f59600090f0).stop 2023-10-18T18:00:49.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f59753a9700 1 -- 172.21.15.72:0/3599418982 shutdown_connections 2023-10-18T18:00:49.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f59753a9700 1 --2- 172.21.15.72:0/3599418982 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f5970062e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:49.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f59753a9700 1 --2- 172.21.15.72:0/3599418982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597014d080 0x7f5970062920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:49.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f59753a9700 1 -- 172.21.15.72:0/3599418982 >> 172.21.15.72:0/3599418982 conn(0x7f597005dc20 msgr2=0x7f597005e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:49.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f59753a9700 1 -- 172.21.15.72:0/3599418982 shutdown_connections 2023-10-18T18:00:49.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.867+0000 7f59753a9700 1 -- 172.21.15.72:0/3599418982 wait complete. 2023-10-18T18:00:49.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59753a9700 1 Processor -- start 2023-10-18T18:00:49.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59753a9700 1 -- start start 2023-10-18T18:00:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59753a9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f597005e860 0x7f597014b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59753a9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f59700c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:49.875 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59753a9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597014d080 0x7f5970144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59753a9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c7e30 con 0x7f597014d080 2023-10-18T18:00:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59753a9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c6900 con 0x7f597005e860 2023-10-18T18:00:49.876 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59753a9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5970161c80 con 0x7f59700c5210 2023-10-18T18:00:49.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59667fc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f59700c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f596effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f597005e860 0x7f597014b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f596effd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f597005e860 0x7f597014b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:37996/0 (socket says 172.21.15.72:37996) 2023-10-18T18:00:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59667fc700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f59700c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:60974/0 (socket says 172.21.15.72:60974) 2023-10-18T18:00:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f596effd700 1 -- 172.21.15.72:0/686746480 learned_addr learned my addr 172.21.15.72:0/686746480 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:49.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f596ffff700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597014d080 0x7f5970144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59667fc700 1 -- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f597005e860 msgr2=0x7f597014b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59667fc700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f597005e860 0x7f597014b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59667fc700 1 -- 172.21.15.72:0/686746480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597014d080 msgr2=0x7f5970144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59667fc700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597014d080 0x7f5970144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:49.879 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f59667fc700 1 -- 172.21.15.72:0/686746480 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5960009370 con 0x7f59700c5210 2023-10-18T18:00:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.871+0000 7f596effd700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f597005e860 0x7f597014b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:00:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.875+0000 7f59667fc700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f59700c6c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f595800e930 tx=0x7f5958009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.875+0000 7f59677fe700 1 -- 172.21.15.72:0/686746480 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958009450 con 0x7f59700c5210 2023-10-18T18:00:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.875+0000 7f59677fe700 1 -- 172.21.15.72:0/686746480 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5958012600 con 0x7f59700c5210 2023-10-18T18:00:49.880 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.875+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59701536f0 con 0x7f59700c5210 2023-10-18T18:00:49.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.875+0000 7f59677fe700 1 -- 172.21.15.72:0/686746480 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958012980 con 0x7f59700c5210 2023-10-18T18:00:49.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.875+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59700c5e70 con 0x7f59700c5210 2023-10-18T18:00:49.881 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.875+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59701617c0 con 0x7f59700c5210 2023-10-18T18:00:49.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.875+0000 7f59677fe700 1 -- 172.21.15.72:0/686746480 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f5958012d30 con 0x7f59700c5210 2023-10-18T18:00:49.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.879+0000 7f59677fe700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f595005a510 0x7f595005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:49.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.879+0000 7f59677fe700 1 -- 172.21.15.72:0/686746480 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f5958074c60 con 0x7f59700c5210 2023-10-18T18:00:49.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.879+0000 7f596effd700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f595005a510 0x7f595005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:49.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.879+0000 7f596effd700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f595005a510 0x7f595005c9d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f596000b3d0 tx=0x7f5960017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:49.891 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:49.887+0000 7f59677fe700 1 -- 172.21.15.72:0/686746480 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f5958048270 con 0x7f59700c5210 2023-10-18T18:00:50.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.079+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f597006c190 con 0x7f59700c5210 2023-10-18T18:00:50.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.083+0000 7f59677fe700 1 -- 172.21.15.72:0/686746480 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f595804b890 con 0x7f59700c5210 2023-10-18T18:00:50.085 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:50.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f595005a510 msgr2=0x7f595005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:50.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f595005a510 0x7f595005c9d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f596000b3d0 tx=0x7f5960017000).stop 2023-10-18T18:00:50.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 msgr2=0x7f59700c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f59700c6c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f595800e930 tx=0x7f5958009060).stop 2023-10-18T18:00:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 shutdown_connections 2023-10-18T18:00:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f595005a510 0x7f595005c9d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f597014d080 0x7f5970144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:50.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f59700c5210 0x7f59700c6c70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 --2- 172.21.15.72:0/686746480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f597005e860 0x7f597014b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 >> 172.21.15.72:0/686746480 conn(0x7f597005dc20 msgr2=0x7f5970149510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 shutdown_connections 2023-10-18T18:00:50.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:50.087+0000 7f59753a9700 1 -- 172.21.15.72:0/686746480 wait complete. 2023-10-18T18:00:50.109 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:53.111 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:53.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.271+0000 7fa50f25b700 1 Processor -- start 2023-10-18T18:00:53.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.271+0000 7fa50f25b700 1 -- start start 2023-10-18T18:00:53.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50f25b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa508067ff0 0x7fa5080683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:53.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50f25b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa50814d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:53.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50f25b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa508065cf0 con 0x7fa5080666b0 2023-10-18T18:00:53.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50f25b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa508150350 con 0x7fa508067ff0 2023-10-18T18:00:53.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50f25b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080650e0 con 0x7fa50814cfa0 2023-10-18T18:00:53.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa4fffff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fa5080666b0 0x7fa5080c1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:50714/0 (socket says 172.21.15.72:50714) 2023-10-18T18:00:53.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa4fffff700 1 -- 172.21.15.72:0/689584745 learned_addr learned my addr 172.21.15.72:0/689584745 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:53.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50cff7700 1 --2- 172.21.15.72:0/689584745 >> v2:172.21.15.179:3300/0 conn(0x7fa508067ff0 0x7fa5080683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:53.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50d7f8700 1 --2- 172.21.15.72:0/689584745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa50814d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:53.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa4ff7fe700 1 -- 172.21.15.72:0/689584745 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473205617 0 0) 0x7fa508065cf0 con 0x7fa5080666b0 2023-10-18T18:00:53.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa4ff7fe700 1 -- 172.21.15.72:0/689584745 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f00033f0 con 0x7fa5080666b0 2023-10-18T18:00:53.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50d7f8700 1 -- 172.21.15.72:0/689584745 >> v2:172.21.15.179:3300/0 conn(0x7fa508067ff0 msgr2=0x7fa5080683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:53.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50d7f8700 1 --2- 172.21.15.72:0/689584745 >> v2:172.21.15.179:3300/0 conn(0x7fa508067ff0 0x7fa5080683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:53.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50d7f8700 1 -- 172.21.15.72:0/689584745 >> v1:172.21.15.72:6789/0 conn(0x7fa5080666b0 legacy=0x7fa5080c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:53.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50d7f8700 1 -- 172.21.15.72:0/689584745 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080653c0 con 0x7fa50814cfa0 2023-10-18T18:00:53.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa50d7f8700 1 --2- 172.21.15.72:0/689584745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa50814d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fa500009e60 tx=0x7fa500009780).ready entity=mon.2 client_cookie=320f38a14261b9d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:53.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa4ff7fe700 1 -- 172.21.15.72:0/689584745 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50000e070 con 0x7fa50814cfa0 2023-10-18T18:00:53.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa4ff7fe700 1 -- 172.21.15.72:0/689584745 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50000bb40 con 0x7fa50814cfa0 2023-10-18T18:00:53.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.275+0000 7fa4ff7fe700 1 -- 172.21.15.72:0/689584745 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500012410 con 0x7fa50814cfa0 2023-10-18T18:00:53.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.279+0000 7fa50f25b700 1 -- 172.21.15.72:0/689584745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 msgr2=0x7fa50814d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:53.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.279+0000 7fa50f25b700 1 --2- 172.21.15.72:0/689584745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa50814d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fa500009e60 tx=0x7fa500009780).stop 2023-10-18T18:00:53.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.279+0000 7fa50f25b700 1 -- 172.21.15.72:0/689584745 shutdown_connections 2023-10-18T18:00:53.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.279+0000 7fa50f25b700 1 --2- 172.21.15.72:0/689584745 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa50814d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:53.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.279+0000 7fa50f25b700 1 --2- 172.21.15.72:0/689584745 >> v2:172.21.15.179:3300/0 conn(0x7fa508067ff0 0x7fa5080683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:53.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.279+0000 7fa50f25b700 1 -- 172.21.15.72:0/689584745 >> 172.21.15.72:0/689584745 conn(0x7fa5080620d0 msgr2=0x7fa5080624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:53.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.279+0000 7fa50f25b700 1 -- 172.21.15.72:0/689584745 shutdown_connections 2023-10-18T18:00:53.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.279+0000 7fa50f25b700 1 -- 172.21.15.72:0/689584745 wait complete. 2023-10-18T18:00:53.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.279+0000 7fa50f25b700 1 Processor -- start 2023-10-18T18:00:53.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50f25b700 1 -- start start 2023-10-18T18:00:53.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50f25b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5080666b0 0x7fa508160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:53.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50f25b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa508067ff0 0x7fa508145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:53.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50f25b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa5080c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:53.288 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50f25b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080bf2d0 con 0x7fa5080666b0 2023-10-18T18:00:53.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50f25b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080bf5b0 con 0x7fa508067ff0 2023-10-18T18:00:53.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50f25b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080bf890 con 0x7fa50814cfa0 2023-10-18T18:00:53.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa4fffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa508067ff0 0x7fa508145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:53.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50d7f8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa5080c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:53.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa4fffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa508067ff0 0x7fa508145910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:38014/0 (socket says 172.21.15.72:38014) 2023-10-18T18:00:53.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50d7f8700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa5080c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:32772/0 (socket says 172.21.15.72:32772) 2023-10-18T18:00:53.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa4fffff700 1 -- 172.21.15.72:0/4016764249 learned_addr learned my addr 172.21.15.72:0/4016764249 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:53.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50cff7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5080666b0 0x7fa508160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:53.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50d7f8700 1 -- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa508067ff0 msgr2=0x7fa508145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:53.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50d7f8700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa508067ff0 0x7fa508145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:53.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50d7f8700 1 -- 172.21.15.72:0/4016764249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5080666b0 msgr2=0x7fa508160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:53.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50d7f8700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5080666b0 0x7fa508160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:53.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50d7f8700 1 -- 172.21.15.72:0/4016764249 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa500009a20 con 0x7fa50814cfa0 2023-10-18T18:00:53.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa50d7f8700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa5080c6a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa500014000 tx=0x7fa500009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:53.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.283+0000 7fa4fcff9700 1 -- 172.21.15.72:0/4016764249 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500012410 con 0x7fa50814cfa0 2023-10-18T18:00:53.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.287+0000 7fa4fcff9700 1 -- 172.21.15.72:0/4016764249 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50001d630 con 0x7fa50814cfa0 2023-10-18T18:00:53.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.287+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5080bfb70 con 0x7fa50814cfa0 2023-10-18T18:00:53.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.287+0000 7fa4fcff9700 1 -- 172.21.15.72:0/4016764249 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50001da10 con 0x7fa50814cfa0 2023-10-18T18:00:53.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.287+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5080c0130 con 0x7fa50814cfa0 2023-10-18T18:00:53.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.287+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50806c190 con 0x7fa50814cfa0 2023-10-18T18:00:53.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.287+0000 7fa4fcff9700 1 -- 172.21.15.72:0/4016764249 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa500003040 con 0x7fa50814cfa0 2023-10-18T18:00:53.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.291+0000 7fa4fcff9700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4f405a510 0x7fa4f405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:53.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.291+0000 7fa4fcff9700 1 -- 172.21.15.72:0/4016764249 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fa500076600 con 0x7fa50814cfa0 2023-10-18T18:00:53.294 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.291+0000 7fa50cff7700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4f405a510 0x7fa4f405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:53.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.291+0000 7fa50cff7700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4f405a510 0x7fa4f405c9d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fa504009c30 tx=0x7fa504009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:53.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.299+0000 7fa4fcff9700 1 -- 172.21.15.72:0/4016764249 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fa500049c10 con 0x7fa50814cfa0 2023-10-18T18:00:53.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.495+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa5080656a0 con 0x7fa50814cfa0 2023-10-18T18:00:53.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.495+0000 7fa4fcff9700 1 -- 172.21.15.72:0/4016764249 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fa50004d230 con 0x7fa50814cfa0 2023-10-18T18:00:53.498 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:53.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4f405a510 msgr2=0x7fa4f405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:53.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4f405a510 0x7fa4f405c9d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fa504009c30 tx=0x7fa504009060).stop 2023-10-18T18:00:53.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 msgr2=0x7fa5080c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:53.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa5080c6a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa500014000 tx=0x7fa500009780).stop 2023-10-18T18:00:53.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 shutdown_connections 2023-10-18T18:00:53.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4f405a510 0x7fa4f405c9d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:53.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa50814cfa0 0x7fa5080c6a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:53.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa508067ff0 0x7fa508145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:53.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 --2- 172.21.15.72:0/4016764249 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5080666b0 0x7fa508160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:53.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 >> 172.21.15.72:0/4016764249 conn(0x7fa5080620d0 msgr2=0x7fa50814c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:53.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 shutdown_connections 2023-10-18T18:00:53.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:53.499+0000 7fa50f25b700 1 -- 172.21.15.72:0/4016764249 wait complete. 2023-10-18T18:00:53.523 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:55.314 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~144s 2023-10-18T18:00:55.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~90s 2023-10-18T18:00:55.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~44s 2023-10-18T18:00:55.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~70s 2023-10-18T18:00:55.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~57s 2023-10-18T18:00:55.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~103s 2023-10-18T18:00:56.525 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:00:56.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.679+0000 7f411e09b700 1 Processor -- start 2023-10-18T18:00:56.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.679+0000 7f411e09b700 1 -- start start 2023-10-18T18:00:56.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f411e09b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180bca50 0x7f41180b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:56.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f411e09b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4118065d30 0x7f411814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:56.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f411e09b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41180bc480 con 0x7f41180b9690 2023-10-18T18:00:56.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f411e09b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4118065190 con 0x7f4118065d30 2023-10-18T18:00:56.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f411e09b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41180bb920 con 0x7f41180bca50 2023-10-18T18:00:56.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f410effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180bca50 0x7f41180b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:56.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f41177fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f41180b9690 0x7f4118065620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:60550/0 (socket says 172.21.15.72:60550) 2023-10-18T18:00:56.688 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4117fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f4118065d30 0x7f411814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:56.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f41177fe700 1 -- 172.21.15.72:0/3789400914 learned_addr learned my addr 172.21.15.72:0/3789400914 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:00:56.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4117fff700 1 -- 172.21.15.72:0/3789400914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180bca50 msgr2=0x7f41180b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:00:56.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4117fff700 1 --2- 172.21.15.72:0/3789400914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180bca50 0x7f41180b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:56.689 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4117fff700 1 -- 172.21.15.72:0/3789400914 >> v1:172.21.15.179:6789/0 conn(0x7f41180b9690 legacy=0x7f4118065620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:56.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4117fff700 1 -- 172.21.15.72:0/3789400914 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41180bbbe0 con 0x7f4118065d30 2023-10-18T18:00:56.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4116ffd700 1 -- 172.21.15.72:0/3789400914 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014128936 0 0) 0x7f41180bc480 con 0x7f41180b9690 2023-10-18T18:00:56.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4117fff700 1 --2- 172.21.15.72:0/3789400914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4118065d30 0x7f411814dd50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f4110009e60 tx=0x7f4110009780).ready entity=mon.1 client_cookie=5207a3de9f11948e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:56.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4116ffd700 1 -- 172.21.15.72:0/3789400914 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000e070 con 0x7f4118065d30 2023-10-18T18:00:56.690 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4116ffd700 1 -- 172.21.15.72:0/3789400914 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411000bb60 con 0x7f4118065d30 2023-10-18T18:00:56.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.683+0000 7f4116ffd700 1 -- 172.21.15.72:0/3789400914 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110012410 con 0x7f4118065d30 2023-10-18T18:00:56.691 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:00:56.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.687+0000 7f411e09b700 1 -- 172.21.15.72:0/3789400914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4118065d30 msgr2=0x7f411814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:56.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.687+0000 7f411e09b700 1 --2- 172.21.15.72:0/3789400914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4118065d30 0x7f411814dd50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f4110009e60 tx=0x7f4110009780).stop 2023-10-18T18:00:56.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.687+0000 7f411e09b700 1 -- 172.21.15.72:0/3789400914 shutdown_connections 2023-10-18T18:00:56.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.687+0000 7f411e09b700 1 --2- 172.21.15.72:0/3789400914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4118065d30 0x7f411814dd50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:56.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.687+0000 7f411e09b700 1 --2- 172.21.15.72:0/3789400914 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180bca50 0x7f41180b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:56.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.687+0000 7f411e09b700 1 -- 172.21.15.72:0/3789400914 >> 172.21.15.72:0/3789400914 conn(0x7f41180620d0 msgr2=0x7f41180624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:56.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.687+0000 7f411e09b700 1 -- 172.21.15.72:0/3789400914 shutdown_connections 2023-10-18T18:00:56.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.687+0000 7f411e09b700 1 -- 172.21.15.72:0/3789400914 wait complete. 2023-10-18T18:00:56.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f411e09b700 1 Processor -- start 2023-10-18T18:00:56.697 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:00:56.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f411e09b700 1 -- start start 2023-10-18T18:00:56.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f411e09b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4118065d30 0x7f41180c0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:56.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f411e09b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180b9690 0x7f41180c12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:56.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f411e09b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41180c0900 0x7f41180c1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:56.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f411e09b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411815e7e0 con 0x7f4118065d30 2023-10-18T18:00:56.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f411e09b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411815dbf0 con 0x7f41180c0900 2023-10-18T18:00:56.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f411e09b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4118146600 con 0x7f41180b9690 2023-10-18T18:00:56.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f410effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180b9690 0x7f41180c12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:56.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f4117fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41180c0900 0x7f41180c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:56.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f41177fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4118065d30 0x7f41180c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:56.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f410effd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180b9690 0x7f41180c12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:32792/0 (socket says 172.21.15.72:32792) 2023-10-18T18:00:56.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.691+0000 7f410effd700 1 -- 172.21.15.72:0/2609286191 learned_addr learned my addr 172.21.15.72:0/2609286191 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:00:56.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f4117fff700 1 -- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180b9690 msgr2=0x7f41180c12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:56.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f4117fff700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180b9690 0x7f41180c12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:56.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f4117fff700 1 -- 172.21.15.72:0/2609286191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4118065d30 msgr2=0x7f41180c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:56.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f4117fff700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4118065d30 0x7f41180c0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:56.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f4117fff700 1 -- 172.21.15.72:0/2609286191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110009a20 con 0x7f41180c0900 2023-10-18T18:00:56.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f4117fff700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41180c0900 0x7f41180c1790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f4110014000 tx=0x7f4110009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:56.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f410ffff700 1 -- 172.21.15.72:0/2609286191 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411001d630 con 0x7f41180c0900 2023-10-18T18:00:56.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4118148870 con 0x7f41180c0900 2023-10-18T18:00:56.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f410ffff700 1 -- 172.21.15.72:0/2609286191 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411001d790 con 0x7f41180c0900 2023-10-18T18:00:56.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f410ffff700 1 -- 172.21.15.72:0/2609286191 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411001db50 con 0x7f41180c0900 2023-10-18T18:00:56.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411815e190 con 0x7f41180c0900 2023-10-18T18:00:56.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.695+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411815e950 con 0x7f41180c0900 2023-10-18T18:00:56.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.699+0000 7f410ffff700 1 -- 172.21.15.72:0/2609286191 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4110003040 con 0x7f41180c0900 2023-10-18T18:00:56.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.699+0000 7f410ffff700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f40f805a4e0 0x7f40f805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:00:56.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.703+0000 7f410ffff700 1 -- 172.21.15.72:0/2609286191 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f4110076fe0 con 0x7f41180c0900 2023-10-18T18:00:56.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.703+0000 7f41177fe700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f40f805a4e0 0x7f40f805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:00:56.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.703+0000 7f41177fe700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f40f805a4e0 0x7f40f805c9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f41000011e0 tx=0x7f4100007bb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:00:56.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.707+0000 7f410ffff700 1 -- 172.21.15.72:0/2609286191 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f411004a5f0 con 0x7f41180c0900 2023-10-18T18:00:56.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.915+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f411806c190 con 0x7f41180c0900 2023-10-18T18:00:56.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.915+0000 7f410ffff700 1 -- 172.21.15.72:0/2609286191 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f411004dc10 con 0x7f41180c0900 2023-10-18T18:00:56.918 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:00:56.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f40f805a4e0 msgr2=0x7f40f805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:56.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f40f805a4e0 0x7f40f805c9a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f41000011e0 tx=0x7f4100007bb0).stop 2023-10-18T18:00:56.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41180c0900 msgr2=0x7f41180c1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:00:56.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41180c0900 0x7f41180c1790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f4110014000 tx=0x7f4110009780).stop 2023-10-18T18:00:56.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 shutdown_connections 2023-10-18T18:00:56.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f40f805a4e0 0x7f40f805c9a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:56.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41180c0900 0x7f41180c1790 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:56.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f41180b9690 0x7f41180c12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:56.925 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 --2- 172.21.15.72:0/2609286191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4118065d30 0x7f41180c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:00:56.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 >> 172.21.15.72:0/2609286191 conn(0x7f41180620d0 msgr2=0x7f41180624b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:00:56.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 shutdown_connections 2023-10-18T18:00:56.926 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:00:56.919+0000 7f411e09b700 1 -- 172.21.15.72:0/2609286191 wait complete. 2023-10-18T18:00:56.944 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:00:59.945 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:00.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.099+0000 7fc4f85db700 1 Processor -- start 2023-10-18T18:01:00.104 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.099+0000 7fc4f85db700 1 -- start start 2023-10-18T18:01:00.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.103+0000 7fc4f85db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f014cce0 0x7fc4f014d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:00.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.103+0000 7fc4f85db700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc4f0067e50 0x7fc4f00be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:00.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.103+0000 7fc4f85db700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f0065aa0 con 0x7fc4f0068940 2023-10-18T18:01:00.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.103+0000 7fc4f85db700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f0069470 con 0x7fc4f0067e50 2023-10-18T18:01:00.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.103+0000 7fc4f85db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00bd100 con 0x7fc4f014cce0 2023-10-18T18:01:00.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.103+0000 7fc4f6b78700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc4f0068940 0x7fc4f0065f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52798/0 (socket says 172.21.15.72:52798) 2023-10-18T18:01:00.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.103+0000 7fc4f6b78700 1 -- 172.21.15.72:0/3284563222 learned_addr learned my addr 172.21.15.72:0/3284563222 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:01:00.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.103+0000 7fc4f6377700 1 --2- 172.21.15.72:0/3284563222 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f014cce0 0x7fc4f014d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:00.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.103+0000 7fc4f5b76700 1 --2- 172.21.15.72:0/3284563222 >> v2:172.21.15.179:3300/0 conn(0x7fc4f0067e50 0x7fc4f00be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:00.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f6377700 1 -- 172.21.15.72:0/3284563222 >> v2:172.21.15.179:3300/0 conn(0x7fc4f0067e50 msgr2=0x7fc4f00be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:00.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f6377700 1 --2- 172.21.15.72:0/3284563222 >> v2:172.21.15.179:3300/0 conn(0x7fc4f0067e50 0x7fc4f00be320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:00.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f5375700 1 -- 172.21.15.72:0/3284563222 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187986788 0 0) 0x7fc4f0065aa0 con 0x7fc4f0068940 2023-10-18T18:01:00.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f6377700 1 -- 172.21.15.72:0/3284563222 >> v1:172.21.15.72:6789/0 conn(0x7fc4f0068940 legacy=0x7fc4f0065f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:00.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f6377700 1 -- 172.21.15.72:0/3284563222 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f00bce20 con 0x7fc4f014cce0 2023-10-18T18:01:00.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f6377700 1 --2- 172.21.15.72:0/3284563222 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f014cce0 0x7fc4f014d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fc4ec009760 tx=0x7fc4ec009060).ready entity=mon.2 client_cookie=10d75aaa11a58611 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:00.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f5375700 1 -- 172.21.15.72:0/3284563222 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00b590 con 0x7fc4f014cce0 2023-10-18T18:01:00.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f5375700 1 -- 172.21.15.72:0/3284563222 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec00b6f0 con 0x7fc4f014cce0 2023-10-18T18:01:00.112 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f5375700 1 -- 172.21.15.72:0/3284563222 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00ba70 con 0x7fc4f014cce0 2023-10-18T18:01:00.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f85db700 1 -- 172.21.15.72:0/3284563222 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f014cce0 msgr2=0x7fc4f014d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:00.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f85db700 1 --2- 172.21.15.72:0/3284563222 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f014cce0 0x7fc4f014d0c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fc4ec009760 tx=0x7fc4ec009060).stop 2023-10-18T18:01:00.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f85db700 1 -- 172.21.15.72:0/3284563222 shutdown_connections 2023-10-18T18:01:00.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f85db700 1 --2- 172.21.15.72:0/3284563222 >> v2:172.21.15.179:3300/0 conn(0x7fc4f0067e50 0x7fc4f00be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:00.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f85db700 1 --2- 172.21.15.72:0/3284563222 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f014cce0 0x7fc4f014d0c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:00.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f85db700 1 -- 172.21.15.72:0/3284563222 >> 172.21.15.72:0/3284563222 conn(0x7fc4f00620d0 msgr2=0x7fc4f00624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:00.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f85db700 1 -- 172.21.15.72:0/3284563222 shutdown_connections 2023-10-18T18:01:00.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.107+0000 7fc4f85db700 1 -- 172.21.15.72:0/3284563222 wait complete. 2023-10-18T18:01:00.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f85db700 1 Processor -- start 2023-10-18T18:01:00.115 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f85db700 1 -- start start 2023-10-18T18:01:00.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f85db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f0067e50 0x7fc4f00ba310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:00.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f85db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f0068940 0x7fc4f0156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:00.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f85db700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4f014cce0 0x7fc4f00bb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:00.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f85db700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0155dd0 con 0x7fc4f014cce0 2023-10-18T18:01:00.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f85db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00bbc40 con 0x7fc4f0067e50 2023-10-18T18:01:00.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f85db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0154b30 con 0x7fc4f0068940 2023-10-18T18:01:00.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6b78700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4f014cce0 0x7fc4f00bb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:00.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6b78700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4f014cce0 0x7fc4f00bb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58408/0 (socket says 172.21.15.72:58408) 2023-10-18T18:01:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6b78700 1 -- 172.21.15.72:0/3402524369 learned_addr learned my addr 172.21.15.72:0/3402524369 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6377700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f0067e50 0x7fc4f00ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f5b76700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f0068940 0x7fc4f0156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6b78700 1 -- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f0068940 msgr2=0x7fc4f0156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:00.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6b78700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f0068940 0x7fc4f0156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:00.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6b78700 1 -- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f0067e50 msgr2=0x7fc4f00ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:00.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6b78700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f0067e50 0x7fc4f00ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:00.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6b78700 1 -- 172.21.15.72:0/3402524369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec0092a0 con 0x7fc4f014cce0 2023-10-18T18:01:00.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4f6b78700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4f014cce0 0x7fc4f00bb030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc4e800c7e0 tx=0x7fc4e8007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:00.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4e67fc700 1 -- 172.21.15.72:0/3402524369 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8005d30 con 0x7fc4f014cce0 2023-10-18T18:01:00.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4e67fc700 1 -- 172.21.15.72:0/3402524369 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e8005e90 con 0x7fc4f014cce0 2023-10-18T18:01:00.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.111+0000 7fc4e67fc700 1 -- 172.21.15.72:0/3402524369 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8007320 con 0x7fc4f014cce0 2023-10-18T18:01:00.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.115+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f00bbe90 con 0x7fc4f014cce0 2023-10-18T18:01:00.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.115+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f01554a0 con 0x7fc4f014cce0 2023-10-18T18:01:00.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.115+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f0155f80 con 0x7fc4f014cce0 2023-10-18T18:01:00.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.115+0000 7fc4e67fc700 1 -- 172.21.15.72:0/3402524369 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc4e80076d0 con 0x7fc4f014cce0 2023-10-18T18:01:00.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.115+0000 7fc4e67fc700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc4d405a510 0x7fc4d405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:00.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.115+0000 7fc4e67fc700 1 -- 172.21.15.72:0/3402524369 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fc4e80749e0 con 0x7fc4f014cce0 2023-10-18T18:01:00.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.119+0000 7fc4f6377700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc4d405a510 0x7fc4d405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:00.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.119+0000 7fc4f6377700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc4d405a510 0x7fc4d405c9d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc4ec00b350 tx=0x7fc4ec017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:00.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.123+0000 7fc4e67fc700 1 -- 172.21.15.72:0/3402524369 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fc4f0155f80 con 0x7fc4f014cce0 2023-10-18T18:01:00.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.319+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc4f0155f80 con 0x7fc4f014cce0 2023-10-18T18:01:00.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.319+0000 7fc4e67fc700 1 -- 172.21.15.72:0/3402524369 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fc4f0155f80 con 0x7fc4f014cce0 2023-10-18T18:01:00.322 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:00.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.319+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc4d405a510 msgr2=0x7fc4d405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:00.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.319+0000 7fc4f85db700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc4d405a510 0x7fc4d405c9d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc4ec00b350 tx=0x7fc4ec017000).stop 2023-10-18T18:01:00.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.319+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4f014cce0 msgr2=0x7fc4f00bb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:00.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.319+0000 7fc4f85db700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4f014cce0 0x7fc4f00bb030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fc4e800c7e0 tx=0x7fc4e8007f90).stop 2023-10-18T18:01:00.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.323+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 shutdown_connections 2023-10-18T18:01:00.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.323+0000 7fc4f85db700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc4d405a510 0x7fc4d405c9d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:00.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.323+0000 7fc4f85db700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc4f014cce0 0x7fc4f00bb030 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:00.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.323+0000 7fc4f85db700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc4f0068940 0x7fc4f0156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:00.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.323+0000 7fc4f85db700 1 --2- 172.21.15.72:0/3402524369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f0067e50 0x7fc4f00ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:00.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.323+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 >> 172.21.15.72:0/3402524369 conn(0x7fc4f00620d0 msgr2=0x7fc4f0065f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:00.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.323+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 shutdown_connections 2023-10-18T18:01:00.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:00.323+0000 7fc4f85db700 1 -- 172.21.15.72:0/3402524369 wait complete. 2023-10-18T18:01:00.343 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:01.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~151s 2023-10-18T18:01:01.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~96s 2023-10-18T18:01:01.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~50s 2023-10-18T18:01:01.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~76s 2023-10-18T18:01:01.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~63s 2023-10-18T18:01:01.627 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~110s 2023-10-18T18:01:03.344 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:03.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.494+0000 7fc772439700 1 Processor -- start 2023-10-18T18:01:03.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc772439700 1 -- start start 2023-10-18T18:01:03.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc772439700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0c2bf0 0x7fc76c065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:03.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc772439700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc76c0bd1c0 0x7fc76c14ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:03.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc772439700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c0be2e0 con 0x7fc76c0658a0 2023-10-18T18:01:03.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc772439700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c0bda80 con 0x7fc76c0bd1c0 2023-10-18T18:01:03.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc772439700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c06a7c0 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc771c38700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fc76c0bd1c0 0x7fc76c14ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:03.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc770c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0c2bf0 0x7fc76c065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:03.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc771437700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fc76c0658a0 0x7fc76c0630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:34394/0 (socket says 172.21.15.72:34394) 2023-10-18T18:01:03.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc771437700 1 -- 172.21.15.72:0/1113597300 learned_addr learned my addr 172.21.15.72:0/1113597300 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:01:03.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc770c36700 1 -- 172.21.15.72:0/1113597300 >> v2:172.21.15.72:3300/0 conn(0x7fc76c0bd1c0 msgr2=0x7fc76c14ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:03.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc770c36700 1 --2- 172.21.15.72:0/1113597300 >> v2:172.21.15.72:3300/0 conn(0x7fc76c0bd1c0 0x7fc76c14ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:03.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc770c36700 1 -- 172.21.15.72:0/1113597300 >> v1:172.21.15.179:6789/0 conn(0x7fc76c0658a0 legacy=0x7fc76c0630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:03.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc763fff700 1 -- 172.21.15.72:0/1113597300 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648753129 0 0) 0x7fc76c0be2e0 con 0x7fc76c0658a0 2023-10-18T18:01:03.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc770c36700 1 -- 172.21.15.72:0/1113597300 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c0be770 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc770c36700 1 --2- 172.21.15.72:0/1113597300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0c2bf0 0x7fc76c065c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fc75c009760 tx=0x7fc75c009060).ready entity=mon.2 client_cookie=44f52049de623f13 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:03.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.498+0000 7fc763fff700 1 -- 172.21.15.72:0/1113597300 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00f510 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc763fff700 1 -- 172.21.15.72:0/1113597300 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c00f670 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc763fff700 1 -- 172.21.15.72:0/1113597300 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00f9f0 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc772439700 1 -- 172.21.15.72:0/1113597300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0c2bf0 msgr2=0x7fc76c065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:03.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc772439700 1 --2- 172.21.15.72:0/1113597300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0c2bf0 0x7fc76c065c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fc75c009760 tx=0x7fc75c009060).stop 2023-10-18T18:01:03.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc772439700 1 -- 172.21.15.72:0/1113597300 shutdown_connections 2023-10-18T18:01:03.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc772439700 1 --2- 172.21.15.72:0/1113597300 >> v2:172.21.15.72:3300/0 conn(0x7fc76c0bd1c0 0x7fc76c14ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:03.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc772439700 1 --2- 172.21.15.72:0/1113597300 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0c2bf0 0x7fc76c065c80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:03.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc772439700 1 -- 172.21.15.72:0/1113597300 >> 172.21.15.72:0/1113597300 conn(0x7fc76c062ba0 msgr2=0x7fc76c0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:03.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc772439700 1 -- 172.21.15.72:0/1113597300 shutdown_connections 2023-10-18T18:01:03.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.502+0000 7fc772439700 1 -- 172.21.15.72:0/1113597300 wait complete. 2023-10-18T18:01:03.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc772439700 1 Processor -- start 2023-10-18T18:01:03.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc772439700 1 -- start start 2023-10-18T18:01:03.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc772439700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0658a0 0x7fc76c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:03.520 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc772439700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc76c0bd1c0 0x7fc76c164180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:03.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc772439700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc76c0c2bf0 0x7fc76c0a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:03.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc772439700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c161ba0 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc772439700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c164670 con 0x7fc76c0bd1c0 2023-10-18T18:01:03.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc772439700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c149ac0 con 0x7fc76c0658a0 2023-10-18T18:01:03.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc771c38700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc76c0c2bf0 0x7fc76c0a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:03.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc771c38700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc76c0c2bf0 0x7fc76c0a7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58428/0 (socket says 172.21.15.72:58428) 2023-10-18T18:01:03.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc771c38700 1 -- 172.21.15.72:0/3949544309 learned_addr learned my addr 172.21.15.72:0/3949544309 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:03.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc770c36700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc76c0bd1c0 0x7fc76c164180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:03.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc771437700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0658a0 0x7fc76c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:03.523 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc771c38700 1 -- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0658a0 msgr2=0x7fc76c1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:03.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc771c38700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0658a0 0x7fc76c1616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:03.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc771c38700 1 -- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc76c0bd1c0 msgr2=0x7fc76c164180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:03.524 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc771c38700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc76c0bd1c0 0x7fc76c164180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:03.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.506+0000 7fc771c38700 1 -- 172.21.15.72:0/3949544309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c0092a0 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.510+0000 7fc771c38700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc76c0c2bf0 0x7fc76c0a7300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc76400e900 tx=0x7fc764009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:03.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.510+0000 7fc7617fa700 1 -- 172.21.15.72:0/3949544309 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764012600 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.510+0000 7fc7617fa700 1 -- 172.21.15.72:0/3949544309 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc764012760 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.510+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c14a640 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.510+0000 7fc7617fa700 1 -- 172.21.15.72:0/3949544309 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764012ae0 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.510+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c149d10 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.510+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76c14a7d0 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.510+0000 7fc7617fa700 1 -- 172.21.15.72:0/3949544309 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc76401b400 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.514+0000 7fc7617fa700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc75005a510 0x7fc75005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:03.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.514+0000 7fc7617fa700 1 -- 172.21.15.72:0/3949544309 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fc764074de0 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.514+0000 7fc771437700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc75005a510 0x7fc75005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:03.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.514+0000 7fc771437700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc75005a510 0x7fc75005c9d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fc768001560 tx=0x7fc768007f30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:03.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.518+0000 7fc7617fa700 1 -- 172.21.15.72:0/3949544309 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fc7640482d0 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.718 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.714+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc76c046f00 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.719 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.714+0000 7fc7617fa700 1 -- 172.21.15.72:0/3949544309 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fc764016020 con 0x7fc76c0c2bf0 2023-10-18T18:01:03.719 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:03.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc75005a510 msgr2=0x7fc75005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:03.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc75005a510 0x7fc75005c9d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fc768001560 tx=0x7fc768007f30).stop 2023-10-18T18:01:03.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc76c0c2bf0 msgr2=0x7fc76c0a7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:03.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc76c0c2bf0 0x7fc76c0a7300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc76400e900 tx=0x7fc764009060).stop 2023-10-18T18:01:03.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 shutdown_connections 2023-10-18T18:01:03.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc75005a510 0x7fc75005c9d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:03.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc76c0c2bf0 0x7fc76c0a7300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:03.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc76c0bd1c0 0x7fc76c164180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:03.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 --2- 172.21.15.72:0/3949544309 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc76c0658a0 0x7fc76c1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:03.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 >> 172.21.15.72:0/3949544309 conn(0x7fc76c062ba0 msgr2=0x7fc76c0c0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:03.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 shutdown_connections 2023-10-18T18:01:03.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:03.718+0000 7fc772439700 1 -- 172.21.15.72:0/3949544309 wait complete. 2023-10-18T18:01:03.742 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:06.743 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:06.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.898+0000 7f042f2f1700 1 Processor -- start 2023-10-18T18:01:06.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.898+0000 7f042f2f1700 1 -- start start 2023-10-18T18:01:06.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042f2f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0428067ff0 0x7f04280683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:06.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042f2f1700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f04280666b0 0x7f04280c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:06.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042f2f1700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f04280c2230 0x7f042814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:06.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042f2f1700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0428065cf0 con 0x7f04280666b0 2023-10-18T18:01:06.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042f2f1700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0428150350 con 0x7f04280c2230 2023-10-18T18:01:06.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042f2f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04280650e0 con 0x7f0428067ff0 2023-10-18T18:01:06.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042c88c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f04280666b0 0x7f04280c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:06.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042c88c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f04280666b0 0x7f04280c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58430/0 (socket says 172.21.15.72:58430) 2023-10-18T18:01:06.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042c88c700 1 -- 172.21.15.72:0/1809388125 learned_addr learned my addr 172.21.15.72:0/1809388125 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:06.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042d08d700 1 --2- 172.21.15.72:0/1809388125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0428067ff0 0x7f04280683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:06.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042d88e700 1 --2- 172.21.15.72:0/1809388125 >> v2:172.21.15.179:3300/0 conn(0x7f04280c2230 0x7f042814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:06.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042c88c700 1 -- 172.21.15.72:0/1809388125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0428067ff0 msgr2=0x7f04280683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:06.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042c88c700 1 --2- 172.21.15.72:0/1809388125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0428067ff0 0x7f04280683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:06.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042c88c700 1 -- 172.21.15.72:0/1809388125 >> v2:172.21.15.179:3300/0 conn(0x7f04280c2230 msgr2=0x7f042814cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:06.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042c88c700 1 --2- 172.21.15.72:0/1809388125 >> v2:172.21.15.179:3300/0 conn(0x7f04280c2230 0x7f042814cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:06.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042c88c700 1 -- 172.21.15.72:0/1809388125 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04280653c0 con 0x7f04280666b0 2023-10-18T18:01:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042c88c700 1 --2- 172.21.15.72:0/1809388125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 0x7f04280c1d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f04281510c0 tx=0x7f041c0090f0).ready entity=mon.0 client_cookie=c3273b277c9fa896 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f0427fff700 1 -- 172.21.15.72:0/1809388125 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c00f510 con 0x7f04280666b0 2023-10-18T18:01:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f0427fff700 1 -- 172.21.15.72:0/1809388125 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041c00f670 con 0x7f04280666b0 2023-10-18T18:01:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f0427fff700 1 -- 172.21.15.72:0/1809388125 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c00f9f0 con 0x7f04280666b0 2023-10-18T18:01:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042f2f1700 1 -- 172.21.15.72:0/1809388125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 msgr2=0x7f04280c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.902+0000 7f042f2f1700 1 --2- 172.21.15.72:0/1809388125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 0x7f04280c1d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f04281510c0 tx=0x7f041c0090f0).stop 2023-10-18T18:01:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.906+0000 7f042f2f1700 1 -- 172.21.15.72:0/1809388125 shutdown_connections 2023-10-18T18:01:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.906+0000 7f042f2f1700 1 --2- 172.21.15.72:0/1809388125 >> v2:172.21.15.179:3300/0 conn(0x7f04280c2230 0x7f042814cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.906+0000 7f042f2f1700 1 --2- 172.21.15.72:0/1809388125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 0x7f04280c1d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.906+0000 7f042f2f1700 1 --2- 172.21.15.72:0/1809388125 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f0428067ff0 0x7f04280683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:06.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.906+0000 7f042f2f1700 1 -- 172.21.15.72:0/1809388125 >> 172.21.15.72:0/1809388125 conn(0x7f04280620d0 msgr2=0x7f04280624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:06.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.906+0000 7f042f2f1700 1 -- 172.21.15.72:0/1809388125 shutdown_connections 2023-10-18T18:01:06.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.906+0000 7f042f2f1700 1 -- 172.21.15.72:0/1809388125 wait complete. 2023-10-18T18:01:06.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.906+0000 7f042f2f1700 1 Processor -- start 2023-10-18T18:01:06.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.906+0000 7f042f2f1700 1 -- start start 2023-10-18T18:01:06.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042f2f1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 0x7f04281600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:06.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042f2f1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0428067ff0 0x7f04280c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:06.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042f2f1700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04280c2230 0x7f04280baac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:06.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042f2f1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04281605d0 con 0x7f04280666b0 2023-10-18T18:01:06.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042f2f1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0428161060 con 0x7f0428067ff0 2023-10-18T18:01:06.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042f2f1700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04280ba630 con 0x7f04280c2230 2023-10-18T18:01:06.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d08d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 0x7f04281600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:06.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d08d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 0x7f04281600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:58438/0 (socket says 172.21.15.72:58438) 2023-10-18T18:01:06.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d08d700 1 -- 172.21.15.72:0/3410241715 learned_addr learned my addr 172.21.15.72:0/3410241715 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:06.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d88e700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04280c2230 0x7f04280baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:06.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042c88c700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0428067ff0 0x7f04280c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:06.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d08d700 1 -- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04280c2230 msgr2=0x7f04280baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:06.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d08d700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04280c2230 0x7f04280baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:06.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d08d700 1 -- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0428067ff0 msgr2=0x7f04280c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:06.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d08d700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0428067ff0 0x7f04280c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:06.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d08d700 1 -- 172.21.15.72:0/3410241715 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c009370 con 0x7f04280666b0 2023-10-18T18:01:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042d08d700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 0x7f04281600e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f041800e950 tx=0x7f0418009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f04257fa700 1 -- 172.21.15.72:0/3410241715 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418012600 con 0x7f04280666b0 2023-10-18T18:01:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f04257fa700 1 -- 172.21.15.72:0/3410241715 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0418012760 con 0x7f04280666b0 2023-10-18T18:01:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04280bb4b0 con 0x7f04280666b0 2023-10-18T18:01:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f04257fa700 1 -- 172.21.15.72:0/3410241715 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418012ae0 con 0x7f04280666b0 2023-10-18T18:01:06.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0428160aa0 con 0x7f04280666b0 2023-10-18T18:01:06.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.910+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042806c190 con 0x7f04280666b0 2023-10-18T18:01:06.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.914+0000 7f04257fa700 1 -- 172.21.15.72:0/3410241715 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f041801b400 con 0x7f04280666b0 2023-10-18T18:01:06.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.914+0000 7f04257fa700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f041405a510 0x7f041405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:06.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.914+0000 7f04257fa700 1 -- 172.21.15.72:0/3410241715 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f0418074eb0 con 0x7f04280666b0 2023-10-18T18:01:06.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.914+0000 7f042c88c700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f041405a510 0x7f041405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:06.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.914+0000 7f042c88c700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f041405a510 0x7f041405c9d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f04281510c0 tx=0x7f041c017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:06.927 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:06.922+0000 7f04257fa700 1 -- 172.21.15.72:0/3410241715 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f0418048460 con 0x7f04280666b0 2023-10-18T18:01:07.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.118+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f04280656a0 con 0x7f04280666b0 2023-10-18T18:01:07.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.118+0000 7f04257fa700 1 -- 172.21.15.72:0/3410241715 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f0418016020 con 0x7f04280666b0 2023-10-18T18:01:07.122 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:07.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f041405a510 msgr2=0x7f041405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:07.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f041405a510 0x7f041405c9d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f04281510c0 tx=0x7f041c017000).stop 2023-10-18T18:01:07.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 msgr2=0x7f04281600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:07.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 0x7f04281600e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f041800e950 tx=0x7f0418009060).stop 2023-10-18T18:01:07.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 shutdown_connections 2023-10-18T18:01:07.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f041405a510 0x7f041405c9d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:07.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f04280c2230 0x7f04280baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:07.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0428067ff0 0x7f04280c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:07.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 --2- 172.21.15.72:0/3410241715 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04280666b0 0x7f04281600e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:07.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 >> 172.21.15.72:0/3410241715 conn(0x7f04280620d0 msgr2=0x7f04280c4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:07.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 shutdown_connections 2023-10-18T18:01:07.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:07.122+0000 7f042f2f1700 1 -- 172.21.15.72:0/3410241715 wait complete. 2023-10-18T18:01:07.142 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:07.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~157s 2023-10-18T18:01:07.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~102s 2023-10-18T18:01:07.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~57s 2023-10-18T18:01:07.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~83s 2023-10-18T18:01:07.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~70s 2023-10-18T18:01:07.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~116s 2023-10-18T18:01:10.143 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:10.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.302+0000 7fd62e29b700 1 Processor -- start 2023-10-18T18:01:10.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.302+0000 7fd62e29b700 1 -- start start 2023-10-18T18:01:10.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62e29b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd628151e00 0x7fd6280bf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:10.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62e29b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6280b81e0 0x7fd6280b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:10.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62e29b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd628151970 con 0x7fd6280bac90 2023-10-18T18:01:10.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62e29b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6280bf960 con 0x7fd628151e00 2023-10-18T18:01:10.310 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62e29b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280bfb70 con 0x7fd6280b81e0 2023-10-18T18:01:10.311 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd627fff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd6280bac90 0x7fd6280b9f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56226/0 (socket says 172.21.15.72:56226) 2023-10-18T18:01:10.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd627fff700 1 -- 172.21.15.72:0/2341945659 learned_addr learned my addr 172.21.15.72:0/2341945659 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:01:10.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62c838700 1 --2- 172.21.15.72:0/2341945659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6280b81e0 0x7fd6280b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:10.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd61f7fe700 1 --2- 172.21.15.72:0/2341945659 >> v2:172.21.15.72:3300/0 conn(0x7fd628151e00 0x7fd6280bf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:10.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd6277fe700 1 -- 172.21.15.72:0/2341945659 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054073894 0 0) 0x7fd628151970 con 0x7fd6280bac90 2023-10-18T18:01:10.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62c838700 1 -- 172.21.15.72:0/2341945659 >> v2:172.21.15.72:3300/0 conn(0x7fd628151e00 msgr2=0x7fd6280bf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:10.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62c838700 1 --2- 172.21.15.72:0/2341945659 >> v2:172.21.15.72:3300/0 conn(0x7fd628151e00 0x7fd6280bf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:10.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62c838700 1 -- 172.21.15.72:0/2341945659 >> v1:172.21.15.72:6789/0 conn(0x7fd6280bac90 legacy=0x7fd6280b9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:10.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62c838700 1 -- 172.21.15.72:0/2341945659 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280bfe50 con 0x7fd6280b81e0 2023-10-18T18:01:10.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62c838700 1 --2- 172.21.15.72:0/2341945659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6280b81e0 0x7fd6280b8650 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd620009760 tx=0x7fd620009060).ready entity=mon.2 client_cookie=57a8333668d511c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:10.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd6277fe700 1 -- 172.21.15.72:0/2341945659 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000f510 con 0x7fd6280b81e0 2023-10-18T18:01:10.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd6277fe700 1 -- 172.21.15.72:0/2341945659 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62000f670 con 0x7fd6280b81e0 2023-10-18T18:01:10.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd6277fe700 1 -- 172.21.15.72:0/2341945659 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000f9f0 con 0x7fd6280b81e0 2023-10-18T18:01:10.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62e29b700 1 -- 172.21.15.72:0/2341945659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6280b81e0 msgr2=0x7fd6280b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:10.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62e29b700 1 --2- 172.21.15.72:0/2341945659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6280b81e0 0x7fd6280b8650 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd620009760 tx=0x7fd620009060).stop 2023-10-18T18:01:10.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.306+0000 7fd62e29b700 1 -- 172.21.15.72:0/2341945659 shutdown_connections 2023-10-18T18:01:10.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 --2- 172.21.15.72:0/2341945659 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd6280b81e0 0x7fd6280b8650 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:10.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 --2- 172.21.15.72:0/2341945659 >> v2:172.21.15.72:3300/0 conn(0x7fd628151e00 0x7fd6280bf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:10.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 -- 172.21.15.72:0/2341945659 >> 172.21.15.72:0/2341945659 conn(0x7fd6280667b0 msgr2=0x7fd628066bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:10.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 -- 172.21.15.72:0/2341945659 shutdown_connections 2023-10-18T18:01:10.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 -- 172.21.15.72:0/2341945659 wait complete. 2023-10-18T18:01:10.316 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 Processor -- start 2023-10-18T18:01:10.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 -- start start 2023-10-18T18:01:10.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6280b81e0 0x7fd62815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:10.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6280bac90 0x7fd6280c0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:10.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628151e00 0x7fd6280a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:10.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd628161600 con 0x7fd6280bac90 2023-10-18T18:01:10.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd628162180 con 0x7fd6280b81e0 2023-10-18T18:01:10.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.310+0000 7fd62e29b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62815ef70 con 0x7fd628151e00 2023-10-18T18:01:10.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd61f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6280bac90 0x7fd6280c0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:10.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62c838700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628151e00 0x7fd6280a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:10.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd61f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6280bac90 0x7fd6280c0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45020/0 (socket says 172.21.15.72:45020) 2023-10-18T18:01:10.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd61f7fe700 1 -- 172.21.15.72:0/1451486768 learned_addr learned my addr 172.21.15.72:0/1451486768 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:10.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd627fff700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6280b81e0 0x7fd62815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:10.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62c838700 1 -- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6280b81e0 msgr2=0x7fd62815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:10.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62c838700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6280b81e0 0x7fd62815ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:10.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62c838700 1 -- 172.21.15.72:0/1451486768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6280bac90 msgr2=0x7fd6280c0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:10.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62c838700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6280bac90 0x7fd6280c0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:10.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62c838700 1 -- 172.21.15.72:0/1451486768 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6200092a0 con 0x7fd628151e00 2023-10-18T18:01:10.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62c838700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628151e00 0x7fd6280a9ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd620014000 tx=0x7fd620017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:10.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd624ff9700 1 -- 172.21.15.72:0/1451486768 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000f7e0 con 0x7fd628151e00 2023-10-18T18:01:10.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd624ff9700 1 -- 172.21.15.72:0/1451486768 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62000f940 con 0x7fd628151e00 2023-10-18T18:01:10.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6280c13d0 con 0x7fd628151e00 2023-10-18T18:01:10.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd624ff9700 1 -- 172.21.15.72:0/1451486768 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62001b660 con 0x7fd628151e00 2023-10-18T18:01:10.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd628161850 con 0x7fd628151e00 2023-10-18T18:01:10.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.314+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62806c190 con 0x7fd628151e00 2023-10-18T18:01:10.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.318+0000 7fd624ff9700 1 -- 172.21.15.72:0/1451486768 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd62001b9d0 con 0x7fd628151e00 2023-10-18T18:01:10.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.318+0000 7fd624ff9700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd60805a510 0x7fd60805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:10.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.318+0000 7fd624ff9700 1 -- 172.21.15.72:0/1451486768 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fd6200763c0 con 0x7fd628151e00 2023-10-18T18:01:10.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.318+0000 7fd627fff700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd60805a510 0x7fd60805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:10.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.322+0000 7fd627fff700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd60805a510 0x7fd60805c9d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fd618001560 tx=0x7fd618007f30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:10.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.330+0000 7fd624ff9700 1 -- 172.21.15.72:0/1451486768 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fd620049910 con 0x7fd628151e00 2023-10-18T18:01:10.528 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.522+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd628151320 con 0x7fd628151e00 2023-10-18T18:01:10.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.526+0000 7fd624ff9700 1 -- 172.21.15.72:0/1451486768 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fd6200290e0 con 0x7fd628151e00 2023-10-18T18:01:10.529 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:10.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd60805a510 msgr2=0x7fd60805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:10.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd60805a510 0x7fd60805c9d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fd618001560 tx=0x7fd618007f30).stop 2023-10-18T18:01:10.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628151e00 msgr2=0x7fd6280a9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:10.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628151e00 0x7fd6280a9ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd620014000 tx=0x7fd620017000).stop 2023-10-18T18:01:10.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 shutdown_connections 2023-10-18T18:01:10.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd60805a510 0x7fd60805c9d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:10.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd628151e00 0x7fd6280a9ad0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:10.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd6280bac90 0x7fd6280c0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:10.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 --2- 172.21.15.72:0/1451486768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6280b81e0 0x7fd62815ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:10.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 >> 172.21.15.72:0/1451486768 conn(0x7fd6280667b0 msgr2=0x7fd628066b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:10.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 shutdown_connections 2023-10-18T18:01:10.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:10.530+0000 7fd62e29b700 1 -- 172.21.15.72:0/1451486768 wait complete. 2023-10-18T18:01:10.553 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:13.554 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:13.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.714+0000 7f9c25e39700 1 Processor -- start 2023-10-18T18:01:13.720 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.714+0000 7f9c25e39700 1 -- start start 2023-10-18T18:01:13.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25e39700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9c200bd1c0 0x7f9c20065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:13.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25e39700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c200be2e0 con 0x7f9c200658a0 2023-10-18T18:01:13.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25e39700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c200be770 con 0x7f9c200c2bf0 2023-10-18T18:01:13.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25e39700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c200bda80 con 0x7f9c200bd1c0 2023-10-18T18:01:13.723 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c24e37700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f9c200658a0 0x7f9c200630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56236/0 (socket says 172.21.15.72:56236) 2023-10-18T18:01:13.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25638700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9c200bd1c0 0x7f9c20065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:13.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c24e37700 1 -- 172.21.15.72:0/1032779702 learned_addr learned my addr 172.21.15.72:0/1032779702 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:01:13.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c17fff700 1 --1- 172.21.15.72:0/1032779702 >> v1:172.21.15.179:6789/0 conn(0x7f9c200c2bf0 0x7f9c2014ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:32920/0 (socket says 172.21.15.72:32920) 2023-10-18T18:01:13.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25638700 1 -- 172.21.15.72:0/1032779702 >> v1:172.21.15.179:6789/0 conn(0x7f9c200c2bf0 legacy=0x7f9c2014ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:13.724 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25638700 1 -- 172.21.15.72:0/1032779702 >> v1:172.21.15.72:6789/0 conn(0x7f9c200658a0 legacy=0x7f9c200630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:13.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c177fe700 1 -- 172.21.15.72:0/1032779702 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748034225 0 0) 0x7f9c200be770 con 0x7f9c200c2bf0 2023-10-18T18:01:13.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25638700 1 -- 172.21.15.72:0/1032779702 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c200b9580 con 0x7f9c200bd1c0 2023-10-18T18:01:13.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25638700 1 --2- 172.21.15.72:0/1032779702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200bd1c0 0x7f9c20065c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f9c18009e90 tx=0x7f9c18009780).ready entity=mon.0 client_cookie=e968e491a54c481d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:13.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c177fe700 1 -- 172.21.15.72:0/1032779702 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1800e070 con 0x7f9c200bd1c0 2023-10-18T18:01:13.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c177fe700 1 -- 172.21.15.72:0/1032779702 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1800bbf0 con 0x7f9c200bd1c0 2023-10-18T18:01:13.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c177fe700 1 -- 172.21.15.72:0/1032779702 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c18012410 con 0x7f9c200bd1c0 2023-10-18T18:01:13.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25e39700 1 -- 172.21.15.72:0/1032779702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200bd1c0 msgr2=0x7f9c20065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:13.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25e39700 1 --2- 172.21.15.72:0/1032779702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200bd1c0 0x7f9c20065c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f9c18009e90 tx=0x7f9c18009780).stop 2023-10-18T18:01:13.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25e39700 1 -- 172.21.15.72:0/1032779702 shutdown_connections 2023-10-18T18:01:13.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25e39700 1 --2- 172.21.15.72:0/1032779702 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200bd1c0 0x7f9c20065c80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:13.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.718+0000 7f9c25e39700 1 -- 172.21.15.72:0/1032779702 >> 172.21.15.72:0/1032779702 conn(0x7f9c20062ba0 msgr2=0x7f9c200c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:13.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 -- 172.21.15.72:0/1032779702 shutdown_connections 2023-10-18T18:01:13.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 -- 172.21.15.72:0/1032779702 wait complete. 2023-10-18T18:01:13.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 Processor -- start 2023-10-18T18:01:13.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 -- start start 2023-10-18T18:01:13.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c200658a0 0x7f9c20067d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:13.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c200bd1c0 0x7f9c20160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:13.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200c2bf0 0x7f9c2014d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:13.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201601e0 con 0x7f9c200c2bf0 2023-10-18T18:01:13.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c200a4ce0 con 0x7f9c200bd1c0 2023-10-18T18:01:13.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25e39700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c200a4fc0 con 0x7f9c200658a0 2023-10-18T18:01:13.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25638700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200c2bf0 0x7f9c2014d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:13.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.722+0000 7f9c25638700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200c2bf0 0x7f9c2014d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45046/0 (socket says 172.21.15.72:45046) 2023-10-18T18:01:13.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25638700 1 -- 172.21.15.72:0/2093774571 learned_addr learned my addr 172.21.15.72:0/2093774571 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:13.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c24e37700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c200658a0 0x7f9c20067d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:13.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c17fff700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c200bd1c0 0x7f9c20160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:13.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25638700 1 -- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c200658a0 msgr2=0x7f9c20067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:13.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25638700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c200658a0 0x7f9c20067d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:13.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25638700 1 -- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c200bd1c0 msgr2=0x7f9c20160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:13.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25638700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c200bd1c0 0x7f9c20160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:13.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25638700 1 -- 172.21.15.72:0/2093774571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c18009a20 con 0x7f9c200c2bf0 2023-10-18T18:01:13.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25638700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200c2bf0 0x7f9c2014d660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f9c180094d0 tx=0x7f9c200ba290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:13.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c14ff9700 1 -- 172.21.15.72:0/2093774571 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1801d700 con 0x7f9c200c2bf0 2023-10-18T18:01:13.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c14ff9700 1 -- 172.21.15.72:0/2093774571 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1801d860 con 0x7f9c200c2bf0 2023-10-18T18:01:13.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c200a52a0 con 0x7f9c200c2bf0 2023-10-18T18:01:13.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c14ff9700 1 -- 172.21.15.72:0/2093774571 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1801dbe0 con 0x7f9c200c2bf0 2023-10-18T18:01:13.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2015fb20 con 0x7f9c200c2bf0 2023-10-18T18:01:13.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.726+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c20066860 con 0x7f9c200c2bf0 2023-10-18T18:01:13.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.730+0000 7f9c14ff9700 1 -- 172.21.15.72:0/2093774571 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9c18027460 con 0x7f9c200c2bf0 2023-10-18T18:01:13.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.730+0000 7f9c14ff9700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9c0c05a510 0x7f9c0c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:13.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.730+0000 7f9c14ff9700 1 -- 172.21.15.72:0/2093774571 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f9c18076af0 con 0x7f9c200c2bf0 2023-10-18T18:01:13.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.730+0000 7f9c24e37700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9c0c05a510 0x7f9c0c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:13.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.730+0000 7f9c24e37700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9c0c05a510 0x7f9c0c05c9d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f9c1c007fd0 tx=0x7f9c1c007d30).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:13.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.738+0000 7f9c14ff9700 1 -- 172.21.15.72:0/2093774571 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f9c20066860 con 0x7f9c200c2bf0 2023-10-18T18:01:13.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.930+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9c20066860 con 0x7f9c200c2bf0 2023-10-18T18:01:13.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.930+0000 7f9c14ff9700 1 -- 172.21.15.72:0/2093774571 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f9c20066860 con 0x7f9c200c2bf0 2023-10-18T18:01:13.935 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:13.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9c0c05a510 msgr2=0x7f9c0c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:13.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9c0c05a510 0x7f9c0c05c9d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f9c1c007fd0 tx=0x7f9c1c007d30).stop 2023-10-18T18:01:13.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200c2bf0 msgr2=0x7f9c2014d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:13.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200c2bf0 0x7f9c2014d660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f9c180094d0 tx=0x7f9c200ba290).stop 2023-10-18T18:01:13.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 shutdown_connections 2023-10-18T18:01:13.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9c0c05a510 0x7f9c0c05c9d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:13.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9c200c2bf0 0x7f9c2014d660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:13.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c200bd1c0 0x7f9c20160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:13.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 --2- 172.21.15.72:0/2093774571 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9c200658a0 0x7f9c20067d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:13.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 >> 172.21.15.72:0/2093774571 conn(0x7f9c20062ba0 msgr2=0x7f9c200bfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:13.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 shutdown_connections 2023-10-18T18:01:13.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:13.934+0000 7f9c25e39700 1 -- 172.21.15.72:0/2093774571 wait complete. 2023-10-18T18:01:13.955 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:14.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~163s 2023-10-18T18:01:14.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~109s 2023-10-18T18:01:14.250 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~63s 2023-10-18T18:01:14.250 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~89s 2023-10-18T18:01:14.250 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~76s 2023-10-18T18:01:14.250 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~122s 2023-10-18T18:01:16.956 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:17.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29198e4700 1 Processor -- start 2023-10-18T18:01:17.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29198e4700 1 -- start start 2023-10-18T18:01:17.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29198e4700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2914067d30 0x7f29140630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:17.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29198e4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f29140635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:17.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29198e4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2914068910 con 0x7f291414d170 2023-10-18T18:01:17.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29198e4700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2914150c10 con 0x7f2914067d30 2023-10-18T18:01:17.124 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29198e4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2914150780 con 0x7f2914068180 2023-10-18T18:01:17.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29190e3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f29140635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:17.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f2913fff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f2914067d30 0x7f29140630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:17.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29190e3700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f29140635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:54802/0 (socket says 172.21.15.72:54802) 2023-10-18T18:01:17.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29190e3700 1 -- 172.21.15.72:0/474801308 learned_addr learned my addr 172.21.15.72:0/474801308 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:17.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29188e2700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f291414d170 0x7f291406a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:32926/0 (socket says 172.21.15.72:32926) 2023-10-18T18:01:17.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29190e3700 1 -- 172.21.15.72:0/474801308 >> v2:172.21.15.72:3300/0 conn(0x7f2914067d30 msgr2=0x7f29140630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:17.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29190e3700 1 --2- 172.21.15.72:0/474801308 >> v2:172.21.15.72:3300/0 conn(0x7f2914067d30 0x7f29140630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:17.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29190e3700 1 -- 172.21.15.72:0/474801308 >> v1:172.21.15.179:6789/0 conn(0x7f291414d170 legacy=0x7f291406a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:17.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.118+0000 7f29190e3700 1 -- 172.21.15.72:0/474801308 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2914068630 con 0x7f2914068180 2023-10-18T18:01:17.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29190e3700 1 --2- 172.21.15.72:0/474801308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f29140635d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f290c009f20 tx=0x7f290c009840).ready entity=mon.2 client_cookie=ab039531c464316a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:17.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29137fe700 1 -- 172.21.15.72:0/474801308 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00e070 con 0x7f2914068180 2023-10-18T18:01:17.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29137fe700 1 -- 172.21.15.72:0/474801308 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c00ba60 con 0x7f2914068180 2023-10-18T18:01:17.128 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29137fe700 1 -- 172.21.15.72:0/474801308 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00bde0 con 0x7f2914068180 2023-10-18T18:01:17.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29198e4700 1 -- 172.21.15.72:0/474801308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 msgr2=0x7f29140635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:17.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29198e4700 1 --2- 172.21.15.72:0/474801308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f29140635d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f290c009f20 tx=0x7f290c009840).stop 2023-10-18T18:01:17.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29198e4700 1 -- 172.21.15.72:0/474801308 shutdown_connections 2023-10-18T18:01:17.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29198e4700 1 --2- 172.21.15.72:0/474801308 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f29140635d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:17.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29198e4700 1 --2- 172.21.15.72:0/474801308 >> v2:172.21.15.72:3300/0 conn(0x7f2914067d30 0x7f29140630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:17.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29198e4700 1 -- 172.21.15.72:0/474801308 >> 172.21.15.72:0/474801308 conn(0x7f2914062ba0 msgr2=0x7f29140c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:17.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29198e4700 1 -- 172.21.15.72:0/474801308 shutdown_connections 2023-10-18T18:01:17.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.122+0000 7f29198e4700 1 -- 172.21.15.72:0/474801308 wait complete. 2023-10-18T18:01:17.130 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 Processor -- start 2023-10-18T18:01:17.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 -- start start 2023-10-18T18:01:17.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2914067d30 0x7f2914086980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:17.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f2914086e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:17.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f291414d170 0x7f2914166170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:17.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29140bc3a0 con 0x7f2914067d30 2023-10-18T18:01:17.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914067670 con 0x7f291414d170 2023-10-18T18:01:17.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2914165ce0 con 0x7f2914068180 2023-10-18T18:01:17.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29188e2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2914067d30 0x7f2914086980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:17.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29188e2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2914067d30 0x7f2914086980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45072/0 (socket says 172.21.15.72:45072) 2023-10-18T18:01:17.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29188e2700 1 -- 172.21.15.72:0/1046026454 learned_addr learned my addr 172.21.15.72:0/1046026454 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:17.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29190e3700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f291414d170 0x7f2914166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:17.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f2913fff700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f2914086e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:17.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29188e2700 1 -- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 msgr2=0x7f2914086e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:17.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29188e2700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f2914086e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:17.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29188e2700 1 -- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f291414d170 msgr2=0x7f2914166170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:17.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29188e2700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f291414d170 0x7f2914166170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:17.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29188e2700 1 -- 172.21.15.72:0/1046026454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c009ae0 con 0x7f2914067d30 2023-10-18T18:01:17.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29190e3700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f291414d170 0x7f2914166170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:01:17.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f2913fff700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f2914086e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:01:17.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29188e2700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2914067d30 0x7f2914086980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f290800c410 tx=0x7f2908007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:17.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f2910ff9700 1 -- 172.21.15.72:0/1046026454 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080072e0 con 0x7f2914067d30 2023-10-18T18:01:17.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f2910ff9700 1 -- 172.21.15.72:0/1046026454 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908007440 con 0x7f2914067d30 2023-10-18T18:01:17.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2914165180 con 0x7f2914067d30 2023-10-18T18:01:17.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f2910ff9700 1 -- 172.21.15.72:0/1046026454 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080058f0 con 0x7f2914067d30 2023-10-18T18:01:17.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.126+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29140bc890 con 0x7f2914067d30 2023-10-18T18:01:17.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.130+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2914046f00 con 0x7f2914067d30 2023-10-18T18:01:17.140 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.130+0000 7f2910ff9700 1 -- 172.21.15.72:0/1046026454 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f2908005ce0 con 0x7f2914067d30 2023-10-18T18:01:17.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.130+0000 7f2910ff9700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f28f805a510 0x7f28f805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:17.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.130+0000 7f2910ff9700 1 -- 172.21.15.72:0/1046026454 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f29080741a0 con 0x7f2914067d30 2023-10-18T18:01:17.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.130+0000 7f2913fff700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f28f805a510 0x7f28f805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:17.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.134+0000 7f2913fff700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f28f805a510 0x7f28f805c9d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f291414e5c0 tx=0x7f2900008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:17.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.138+0000 7f2910ff9700 1 -- 172.21.15.72:0/1046026454 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f2908047730 con 0x7f2914067d30 2023-10-18T18:01:17.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f291414fe50 con 0x7f2914067d30 2023-10-18T18:01:17.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f2910ff9700 1 -- 172.21.15.72:0/1046026454 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f2908018080 con 0x7f2914067d30 2023-10-18T18:01:17.334 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:17.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f28f805a510 msgr2=0x7f28f805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:17.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f28f805a510 0x7f28f805c9d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f291414e5c0 tx=0x7f2900008040).stop 2023-10-18T18:01:17.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2914067d30 msgr2=0x7f2914086980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:17.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2914067d30 0x7f2914086980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f290800c410 tx=0x7f2908007df0).stop 2023-10-18T18:01:17.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 shutdown_connections 2023-10-18T18:01:17.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f28f805a510 0x7f28f805c9d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:17.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f291414d170 0x7f2914166170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:17.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f2914068180 0x7f2914086e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:17.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 --2- 172.21.15.72:0/1046026454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2914067d30 0x7f2914086980 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:17.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.330+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 >> 172.21.15.72:0/1046026454 conn(0x7f2914062ba0 msgr2=0x7f291414ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:17.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.334+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 shutdown_connections 2023-10-18T18:01:17.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:17.334+0000 7f29198e4700 1 -- 172.21.15.72:0/1046026454 wait complete. 2023-10-18T18:01:17.355 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:20.357 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:20.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f90fb9700 1 Processor -- start 2023-10-18T18:01:20.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f90fb9700 1 -- start start 2023-10-18T18:01:20.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f90fb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c150fe0 0x7f1f8c069740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:20.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f90fb9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c068960 con 0x7f1f8c0c0000 2023-10-18T18:01:20.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f90fb9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c064000 con 0x7f1f8c151ad0 2023-10-18T18:01:20.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f90fb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c063ab0 con 0x7f1f8c150fe0 2023-10-18T18:01:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f83fff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f1f8c0c0000 0x7f1f8c1513c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:57036/0 (socket says 172.21.15.72:57036) 2023-10-18T18:01:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f83fff700 1 -- 172.21.15.72:0/120434767 learned_addr learned my addr 172.21.15.72:0/120434767 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:01:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8b7fe700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c150fe0 0x7f1f8c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902608955 0 0) 0x7f1f8c068960 con 0x7f1f8c0c0000 2023-10-18T18:01:20.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f780033f0 con 0x7f1f8c0c0000 2023-10-18T18:01:20.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 557754079 0 0) 0x7f1f780033f0 con 0x7f1f8c0c0000 2023-10-18T18:01:20.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c150fe0 msgr2=0x7f1f8c069740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:20.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c150fe0 0x7f1f8c069740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:20.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 >> v1:172.21.15.179:6789/0 conn(0x7f1f8c151ad0 legacy=0x7f1f8c14dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:20.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0bab60 con 0x7f1f8c0c0000 2023-10-18T18:01:20.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1723103133 0 0) 0x7f1f74001470 con 0x7f1f8c0c0000 2023-10-18T18:01:20.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 >> v1:172.21.15.72:6789/0 conn(0x7f1f8c0c0000 legacy=0x7f1f8c1513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:20.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.522+0000 7f1f8affd700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f78003910 0x7f1f78003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:20.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8affd700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f78004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:20.531 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8affd700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c150fe0 0x7f1f78005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:20.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c063ab0 con 0x7f1f78004690 2023-10-18T18:01:20.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7800b8e0 con 0x7f1f78003910 2023-10-18T18:01:20.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7800ba50 con 0x7f1f8c150fe0 2023-10-18T18:01:20.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8b7fe700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f78003910 0x7f1f78003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:20.532 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f83fff700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f78004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:20.533 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8bfff700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c150fe0 0x7f1f78005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:20.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f83fff700 1 -- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c150fe0 msgr2=0x7f1f78005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:20.534 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f83fff700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c150fe0 0x7f1f78005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:20.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f83fff700 1 -- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f78003910 msgr2=0x7f1f78003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:20.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f83fff700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f78003910 0x7f1f78003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:20.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f83fff700 1 -- 172.21.15.72:0/120434767 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f780033f0 con 0x7f1f78004690 2023-10-18T18:01:20.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8b7fe700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f78003910 0x7f1f78003cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:20.535 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f83fff700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f78004b10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1f8c05f9b0 tx=0x7f1f8c069180).ready entity=mon.0 client_cookie=d88bcdb1a2aefa11 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74007a60 con 0x7f1f78004690 2023-10-18T18:01:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7400ca20 con 0x7f1f78004690 2023-10-18T18:01:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8affd700 1 -- 172.21.15.72:0/120434767 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400cde0 con 0x7f1f78004690 2023-10-18T18:01:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f90fb9700 1 -- 172.21.15.72:0/120434767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 msgr2=0x7f1f78004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f90fb9700 1 --2- 172.21.15.72:0/120434767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f78004b10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1f8c05f9b0 tx=0x7f1f8c069180).stop 2023-10-18T18:01:20.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f8bfff700 1 -- 172.21.15.72:0/120434767 reap_dead start 2023-10-18T18:01:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f90fb9700 1 -- 172.21.15.72:0/120434767 shutdown_connections 2023-10-18T18:01:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f90fb9700 1 -- 172.21.15.72:0/120434767 >> 172.21.15.72:0/120434767 conn(0x7f1f8c062b50 msgr2=0x7f1f8c067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f90fb9700 1 -- 172.21.15.72:0/120434767 shutdown_connections 2023-10-18T18:01:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.526+0000 7f1f90fb9700 1 -- 172.21.15.72:0/120434767 wait complete. 2023-10-18T18:01:20.537 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f90fb9700 1 Processor -- start 2023-10-18T18:01:20.538 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f90fb9700 1 -- start start 2023-10-18T18:01:20.539 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f90fb9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f8c0bb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:20.539 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f90fb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c1485f0 0x7f1f8c160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:20.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f90fb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c1475e0 0x7f1f8c148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:20.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f90fb9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c069c30 con 0x7f1f78004690 2023-10-18T18:01:20.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f90fb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0bb0b0 con 0x7f1f8c1475e0 2023-10-18T18:01:20.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f90fb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c1462f0 con 0x7f1f8c1485f0 2023-10-18T18:01:20.540 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f8b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f8c0bb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:20.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f8b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f8c0bb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57934/0 (socket says 172.21.15.72:57934) 2023-10-18T18:01:20.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f8b7fe700 1 -- 172.21.15.72:0/262145176 learned_addr learned my addr 172.21.15.72:0/262145176 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:20.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f83fff700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c1485f0 0x7f1f8c160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:20.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f8bfff700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c1475e0 0x7f1f8c148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:20.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f8b7fe700 1 -- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c1485f0 msgr2=0x7f1f8c160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:20.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f8b7fe700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c1485f0 0x7f1f8c160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:20.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f8b7fe700 1 -- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c1475e0 msgr2=0x7f1f8c148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:20.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f8b7fe700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c1475e0 0x7f1f8c148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:20.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.530+0000 7f1f8b7fe700 1 -- 172.21.15.72:0/262145176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c147ba0 con 0x7f1f78004690 2023-10-18T18:01:20.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.534+0000 7f1f8b7fe700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f8c0bb490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f1f7c011da0 tx=0x7f1f7c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:20.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.534+0000 7f1f837fe700 1 -- 172.21.15.72:0/262145176 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00fa10 con 0x7f1f78004690 2023-10-18T18:01:20.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.534+0000 7f1f837fe700 1 -- 172.21.15.72:0/262145176 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c00fb70 con 0x7f1f78004690 2023-10-18T18:01:20.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.534+0000 7f1f837fe700 1 -- 172.21.15.72:0/262145176 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c014440 con 0x7f1f78004690 2023-10-18T18:01:20.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.534+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8c148160 con 0x7f1f78004690 2023-10-18T18:01:20.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.534+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c146820 con 0x7f1f78004690 2023-10-18T18:01:20.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.534+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c0b9610 con 0x7f1f78004690 2023-10-18T18:01:20.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.534+0000 7f1f837fe700 1 -- 172.21.15.72:0/262145176 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f1f7c0147b0 con 0x7f1f78004690 2023-10-18T18:01:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.538+0000 7f1f837fe700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1f6c05a540 0x7f1f6c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.538+0000 7f1f837fe700 1 -- 172.21.15.72:0/262145176 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f1f7c075380 con 0x7f1f78004690 2023-10-18T18:01:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.538+0000 7f1f83fff700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1f6c05a540 0x7f1f6c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:20.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.538+0000 7f1f83fff700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1f6c05a540 0x7f1f6c05ca00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f1f7400c9f0 tx=0x7f1f74015000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:20.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.546+0000 7f1f837fe700 1 -- 172.21.15.72:0/262145176 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f1f7c048990 con 0x7f1f78004690 2023-10-18T18:01:20.559 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~169s 2023-10-18T18:01:20.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~115s 2023-10-18T18:01:20.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~69s 2023-10-18T18:01:20.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~95s 2023-10-18T18:01:20.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~82s 2023-10-18T18:01:20.560 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~129s 2023-10-18T18:01:20.742 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.738+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1f8c046f00 con 0x7f1f78004690 2023-10-18T18:01:20.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.738+0000 7f1f837fe700 1 -- 172.21.15.72:0/262145176 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f1f7c022080 con 0x7f1f78004690 2023-10-18T18:01:20.744 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:20.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1f6c05a540 msgr2=0x7f1f6c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:20.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1f6c05a540 0x7f1f6c05ca00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f1f7400c9f0 tx=0x7f1f74015000).stop 2023-10-18T18:01:20.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 msgr2=0x7f1f8c0bb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:20.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f8c0bb490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f1f7c011da0 tx=0x7f1f7c009060).stop 2023-10-18T18:01:20.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 shutdown_connections 2023-10-18T18:01:20.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f1f6c05a540 0x7f1f6c05ca00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:20.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f8c1475e0 0x7f1f8c148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:20.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f1f8c1485f0 0x7f1f8c160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:20.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 --2- 172.21.15.72:0/262145176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1f78004690 0x7f1f8c0bb490 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:20.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 >> 172.21.15.72:0/262145176 conn(0x7f1f8c062b50 msgr2=0x7f1f8c145940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:20.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 shutdown_connections 2023-10-18T18:01:20.748 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:20.742+0000 7f1f90fb9700 1 -- 172.21.15.72:0/262145176 wait complete. 2023-10-18T18:01:20.765 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:23.767 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:23.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a5947700 1 Processor -- start 2023-10-18T18:01:23.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a5947700 1 -- start start 2023-10-18T18:01:23.931 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a5947700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb6a0065e60 0x7fb6a0066240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:23.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a5947700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb6a0063590 0x7fb6a00685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:23.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a5947700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a0150270 con 0x7fb6a0069ec0 2023-10-18T18:01:23.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a5947700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a0068c10 con 0x7fb6a0065e60 2023-10-18T18:01:23.932 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a5947700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a006a340 con 0x7fb6a0063590 2023-10-18T18:01:23.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a4945700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb6a0065e60 0x7fb6a0066240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:23.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a4945700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fb6a0065e60 0x7fb6a0066240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57940/0 (socket says 172.21.15.72:57940) 2023-10-18T18:01:23.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb6a4945700 1 -- 172.21.15.72:0/133407196 learned_addr learned my addr 172.21.15.72:0/133407196 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:23.934 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.926+0000 7fb697fff700 1 --2- 172.21.15.72:0/133407196 >> v2:172.21.15.179:3300/0 conn(0x7fb6a0063590 0x7fb6a00685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:23.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a4945700 1 -- 172.21.15.72:0/133407196 >> v2:172.21.15.179:3300/0 conn(0x7fb6a0063590 msgr2=0x7fb6a00685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:23.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a4945700 1 --2- 172.21.15.72:0/133407196 >> v2:172.21.15.179:3300/0 conn(0x7fb6a0063590 0x7fb6a00685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:23.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a4945700 1 -- 172.21.15.72:0/133407196 >> v1:172.21.15.179:6789/0 conn(0x7fb6a0069ec0 legacy=0x7fb6a014c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:23.935 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a4945700 1 -- 172.21.15.72:0/133407196 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a014f660 con 0x7fb6a0065e60 2023-10-18T18:01:23.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6977fe700 1 -- 172.21.15.72:0/133407196 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808869194 0 0) 0x7fb6a0150270 con 0x7fb6a0069ec0 2023-10-18T18:01:23.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a4945700 1 --2- 172.21.15.72:0/133407196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0065e60 0x7fb6a0066240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fb6a0066730 tx=0x7fb69c0090f0).ready entity=mon.0 client_cookie=166a716072c9800a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:23.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6977fe700 1 -- 172.21.15.72:0/133407196 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00b690 con 0x7fb6a0065e60 2023-10-18T18:01:23.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6977fe700 1 -- 172.21.15.72:0/133407196 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c00b7f0 con 0x7fb6a0065e60 2023-10-18T18:01:23.936 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6977fe700 1 -- 172.21.15.72:0/133407196 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00bb70 con 0x7fb6a0065e60 2023-10-18T18:01:23.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a5947700 1 -- 172.21.15.72:0/133407196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0065e60 msgr2=0x7fb6a0066240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:23.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a5947700 1 --2- 172.21.15.72:0/133407196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0065e60 0x7fb6a0066240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fb6a0066730 tx=0x7fb69c0090f0).stop 2023-10-18T18:01:23.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a5947700 1 -- 172.21.15.72:0/133407196 shutdown_connections 2023-10-18T18:01:23.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a5947700 1 --2- 172.21.15.72:0/133407196 >> v2:172.21.15.179:3300/0 conn(0x7fb6a0063590 0x7fb6a00685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:23.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a5947700 1 --2- 172.21.15.72:0/133407196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0065e60 0x7fb6a0066240 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:23.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a5947700 1 -- 172.21.15.72:0/133407196 >> 172.21.15.72:0/133407196 conn(0x7fb6a00c2780 msgr2=0x7fb6a00c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:23.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a5947700 1 -- 172.21.15.72:0/133407196 shutdown_connections 2023-10-18T18:01:23.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.930+0000 7fb6a5947700 1 -- 172.21.15.72:0/133407196 wait complete. 2023-10-18T18:01:23.939 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.934+0000 7fb6a5947700 1 Processor -- start 2023-10-18T18:01:23.940 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.934+0000 7fb6a5947700 1 -- start start 2023-10-18T18:01:23.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.934+0000 7fb6a5947700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0063590 0x7fb6a00bb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:23.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.934+0000 7fb6a5947700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6a0065e60 0x7fb6a013d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:23.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.934+0000 7fb6a5947700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0069ec0 0x7fb6a0163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:23.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.934+0000 7fb6a5947700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a0138f10 con 0x7fb6a0063590 2023-10-18T18:01:23.941 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.934+0000 7fb6a5947700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00bc080 con 0x7fb6a0069ec0 2023-10-18T18:01:23.942 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.934+0000 7fb6a5947700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a01600e0 con 0x7fb6a0065e60 2023-10-18T18:01:23.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a4945700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0063590 0x7fb6a00bb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:23.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a4945700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0063590 0x7fb6a00bb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57956/0 (socket says 172.21.15.72:57956) 2023-10-18T18:01:23.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a4945700 1 -- 172.21.15.72:0/3829143770 learned_addr learned my addr 172.21.15.72:0/3829143770 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:23.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a5146700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0069ec0 0x7fb6a0163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:23.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb697fff700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6a0065e60 0x7fb6a013d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:23.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a4945700 1 -- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6a0065e60 msgr2=0x7fb6a013d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:23.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a4945700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6a0065e60 0x7fb6a013d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:23.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a4945700 1 -- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0069ec0 msgr2=0x7fb6a0163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:23.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a4945700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0069ec0 0x7fb6a0163ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:23.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a4945700 1 -- 172.21.15.72:0/3829143770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c009370 con 0x7fb6a0063590 2023-10-18T18:01:23.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a4945700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0063590 0x7fb6a00bb230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fb69c006fd0 tx=0x7fb69c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:23.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb694ff9700 1 -- 172.21.15.72:0/3829143770 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00b960 con 0x7fb6a0063590 2023-10-18T18:01:23.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb694ff9700 1 -- 172.21.15.72:0/3829143770 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c00bac0 con 0x7fb6a0063590 2023-10-18T18:01:23.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb694ff9700 1 -- 172.21.15.72:0/3829143770 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c01c630 con 0x7fb6a0063590 2023-10-18T18:01:23.946 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a00c1ef0 con 0x7fb6a0063590 2023-10-18T18:01:23.947 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a00bbac0 con 0x7fb6a0063590 2023-10-18T18:01:23.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.938+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a0046f00 con 0x7fb6a0063590 2023-10-18T18:01:23.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.942+0000 7fb694ff9700 1 -- 172.21.15.72:0/3829143770 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb69c01ca20 con 0x7fb6a0063590 2023-10-18T18:01:23.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.942+0000 7fb694ff9700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb68c05a510 0x7fb68c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:23.948 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.942+0000 7fb694ff9700 1 -- 172.21.15.72:0/3829143770 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fb69c076540 con 0x7fb6a0063590 2023-10-18T18:01:23.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.942+0000 7fb697fff700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb68c05a510 0x7fb68c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:23.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.942+0000 7fb697fff700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb68c05a510 0x7fb68c05c9d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb6a013e950 tx=0x7fb690008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:23.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:23.950+0000 7fb694ff9700 1 -- 172.21.15.72:0/3829143770 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fb69c049b50 con 0x7fb6a0063590 2023-10-18T18:01:24.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.142+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb6a014f940 con 0x7fb6a0063590 2023-10-18T18:01:24.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.142+0000 7fb694ff9700 1 -- 172.21.15.72:0/3829143770 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fb69c04d170 con 0x7fb6a0063590 2023-10-18T18:01:24.147 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:24.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb68c05a510 msgr2=0x7fb68c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:24.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb68c05a510 0x7fb68c05c9d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb6a013e950 tx=0x7fb690008040).stop 2023-10-18T18:01:24.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0063590 msgr2=0x7fb6a00bb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:24.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0063590 0x7fb6a00bb230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fb69c006fd0 tx=0x7fb69c017000).stop 2023-10-18T18:01:24.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 shutdown_connections 2023-10-18T18:01:24.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb68c05a510 0x7fb68c05c9d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:24.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6a0069ec0 0x7fb6a0163ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:24.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb6a0065e60 0x7fb6a013d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:24.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 --2- 172.21.15.72:0/3829143770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb6a0063590 0x7fb6a00bb230 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:24.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 >> 172.21.15.72:0/3829143770 conn(0x7fb6a00c2780 msgr2=0x7fb6a00c0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:24.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 shutdown_connections 2023-10-18T18:01:24.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:24.146+0000 7fb6a5947700 1 -- 172.21.15.72:0/3829143770 wait complete. 2023-10-18T18:01:24.170 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:26.762 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:01:26.770 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:01:26.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~176s 2023-10-18T18:01:26.870 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~121s 2023-10-18T18:01:26.871 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~76s 2023-10-18T18:01:26.871 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~102s 2023-10-18T18:01:26.871 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~89s 2023-10-18T18:01:26.871 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~135s 2023-10-18T18:01:27.171 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:27.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f8481fb9700 1 Processor -- start 2023-10-18T18:01:27.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f8481fb9700 1 -- start start 2023-10-18T18:01:27.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f8481fb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c067d30 0x7f847c0630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:27.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f8481fb9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f847c068150 0x7f847c0635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:27.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f8481fb9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c068910 con 0x7f847c14d170 2023-10-18T18:01:27.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f8481fb9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847c150c10 con 0x7f847c068150 2023-10-18T18:01:27.332 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f8481fb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c150780 con 0x7f847c067d30 2023-10-18T18:01:27.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f84817b8700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f847c068150 0x7f847c0635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:27.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f84817b8700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f847c068150 0x7f847c0635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57958/0 (socket says 172.21.15.72:57958) 2023-10-18T18:01:27.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f84817b8700 1 -- 172.21.15.72:0/1673456639 learned_addr learned my addr 172.21.15.72:0/1673456639 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:27.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.326+0000 7f847bfff700 1 --2- 172.21.15.72:0/1673456639 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c067d30 0x7f847c0630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:27.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f84817b8700 1 -- 172.21.15.72:0/1673456639 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c067d30 msgr2=0x7f847c0630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:27.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f84817b8700 1 --2- 172.21.15.72:0/1673456639 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c067d30 0x7f847c0630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:27.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f84817b8700 1 -- 172.21.15.72:0/1673456639 >> v1:172.21.15.179:6789/0 conn(0x7f847c14d170 legacy=0x7f847c06a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:27.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f84817b8700 1 -- 172.21.15.72:0/1673456639 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c068630 con 0x7f847c068150 2023-10-18T18:01:27.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f847b7fe700 1 -- 172.21.15.72:0/1673456639 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775270345 0 0) 0x7f847c068910 con 0x7f847c14d170 2023-10-18T18:01:27.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f84817b8700 1 --2- 172.21.15.72:0/1673456639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c068150 0x7f847c0635d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f8474009f20 tx=0x7f8474009840).ready entity=mon.0 client_cookie=161ec742a4dcf910 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:27.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f847b7fe700 1 -- 172.21.15.72:0/1673456639 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847400e070 con 0x7f847c068150 2023-10-18T18:01:27.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f847b7fe700 1 -- 172.21.15.72:0/1673456639 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847400ba60 con 0x7f847c068150 2023-10-18T18:01:27.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f847b7fe700 1 -- 172.21.15.72:0/1673456639 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847400bde0 con 0x7f847c068150 2023-10-18T18:01:27.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f8481fb9700 1 -- 172.21.15.72:0/1673456639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c068150 msgr2=0x7f847c0635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:27.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f8481fb9700 1 --2- 172.21.15.72:0/1673456639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c068150 0x7f847c0635d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f8474009f20 tx=0x7f8474009840).stop 2023-10-18T18:01:27.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f8481fb9700 1 -- 172.21.15.72:0/1673456639 shutdown_connections 2023-10-18T18:01:27.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f8481fb9700 1 --2- 172.21.15.72:0/1673456639 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c068150 0x7f847c0635d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f8481fb9700 1 --2- 172.21.15.72:0/1673456639 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c067d30 0x7f847c0630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f8481fb9700 1 -- 172.21.15.72:0/1673456639 >> 172.21.15.72:0/1673456639 conn(0x7f847c062ba0 msgr2=0x7f847c0c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f8481fb9700 1 -- 172.21.15.72:0/1673456639 shutdown_connections 2023-10-18T18:01:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.330+0000 7f8481fb9700 1 -- 172.21.15.72:0/1673456639 wait complete. 2023-10-18T18:01:27.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.334+0000 7f8481fb9700 1 Processor -- start 2023-10-18T18:01:27.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.334+0000 7f8481fb9700 1 -- start start 2023-10-18T18:01:27.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.334+0000 7f8481fb9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c067d30 0x7f847c164600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:27.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.334+0000 7f8481fb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c068150 0x7f847c14be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:27.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.334+0000 7f8481fb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f847c14d170 0x7f847c160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:27.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.334+0000 7f8481fb9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c14e320 con 0x7f847c067d30 2023-10-18T18:01:27.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.334+0000 7f8481fb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c164af0 con 0x7f847c14d170 2023-10-18T18:01:27.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.334+0000 7f8481fb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c1607c0 con 0x7f847c068150 2023-10-18T18:01:27.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8480fb7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c067d30 0x7f847c164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:27.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8480fb7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c067d30 0x7f847c164600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57970/0 (socket says 172.21.15.72:57970) 2023-10-18T18:01:27.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8480fb7700 1 -- 172.21.15.72:0/2866803030 learned_addr learned my addr 172.21.15.72:0/2866803030 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:27.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f84817b8700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f847c14d170 0x7f847c160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:27.343 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f847bfff700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c068150 0x7f847c14be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:27.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8480fb7700 1 -- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c068150 msgr2=0x7f847c14be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:27.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8480fb7700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c068150 0x7f847c14be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:27.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8480fb7700 1 -- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f847c14d170 msgr2=0x7f847c160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:27.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8480fb7700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f847c14d170 0x7f847c160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:27.344 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8480fb7700 1 -- 172.21.15.72:0/2866803030 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8474009ae0 con 0x7f847c067d30 2023-10-18T18:01:27.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8480fb7700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c067d30 0x7f847c164600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f847000c4e0 tx=0x7f8470007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:27.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8478ff9700 1 -- 172.21.15.72:0/2866803030 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847000ce50 con 0x7f847c067d30 2023-10-18T18:01:27.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8478ff9700 1 -- 172.21.15.72:0/2866803030 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847000f470 con 0x7f847c067d30 2023-10-18T18:01:27.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c14e040 con 0x7f847c067d30 2023-10-18T18:01:27.345 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8478ff9700 1 -- 172.21.15.72:0/2866803030 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847000f830 con 0x7f847c067d30 2023-10-18T18:01:27.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.338+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c0ba5a0 con 0x7f847c067d30 2023-10-18T18:01:27.346 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.342+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c046f00 con 0x7f847c067d30 2023-10-18T18:01:27.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.342+0000 7f8478ff9700 1 -- 172.21.15.72:0/2866803030 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f847000fbe0 con 0x7f847c067d30 2023-10-18T18:01:27.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.342+0000 7f8478ff9700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f846005a510 0x7f846005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:27.347 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.342+0000 7f8478ff9700 1 -- 172.21.15.72:0/2866803030 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f8470073d10 con 0x7f847c067d30 2023-10-18T18:01:27.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.342+0000 7f847bfff700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f846005a510 0x7f846005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:27.348 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.342+0000 7f847bfff700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f846005a510 0x7f846005c9d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f847c0634a0 tx=0x7f8468009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:27.354 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.350+0000 7f8478ff9700 1 -- 172.21.15.72:0/2866803030 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f8470079020 con 0x7f847c067d30 2023-10-18T18:01:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.542+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f847c067660 con 0x7f847c067d30 2023-10-18T18:01:27.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.542+0000 7f8478ff9700 1 -- 172.21.15.72:0/2866803030 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f8470047310 con 0x7f847c067d30 2023-10-18T18:01:27.548 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:27.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f846005a510 msgr2=0x7f846005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:27.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f846005a510 0x7f846005c9d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f847c0634a0 tx=0x7f8468009060).stop 2023-10-18T18:01:27.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c067d30 msgr2=0x7f847c164600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:27.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c067d30 0x7f847c164600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f847000c4e0 tx=0x7f8470007c70).stop 2023-10-18T18:01:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 shutdown_connections 2023-10-18T18:01:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f846005a510 0x7f846005c9d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f847c14d170 0x7f847c160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f847c068150 0x7f847c14be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:27.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 --2- 172.21.15.72:0/2866803030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f847c067d30 0x7f847c164600 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.546+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 >> 172.21.15.72:0/2866803030 conn(0x7f847c062ba0 msgr2=0x7f847c0c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.550+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 shutdown_connections 2023-10-18T18:01:27.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:27.550+0000 7f8481fb9700 1 -- 172.21.15.72:0/2866803030 wait complete. 2023-10-18T18:01:27.573 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:30.575 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:30.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff224d69700 1 Processor -- start 2023-10-18T18:01:30.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff224d69700 1 -- start start 2023-10-18T18:01:30.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff224d69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200b9690 0x7ff2200b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:30.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff224d69700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff220065d30 0x7ff22014dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:30.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff224d69700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2200bc480 con 0x7ff2200bca50 2023-10-18T18:01:30.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff224d69700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff220065190 con 0x7ff220065d30 2023-10-18T18:01:30.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff224d69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200bb920 con 0x7ff2200b9690 2023-10-18T18:01:30.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff21e59c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200b9690 0x7ff2200b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:30.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff21dd9b700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7ff2200bca50 0x7ff220065620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:57072/0 (socket says 172.21.15.72:57072) 2023-10-18T18:01:30.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff21dd9b700 1 -- 172.21.15.72:0/3554451212 learned_addr learned my addr 172.21.15.72:0/3554451212 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:01:30.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.730+0000 7ff21ed9d700 1 --2- 172.21.15.72:0/3554451212 >> v2:172.21.15.179:3300/0 conn(0x7ff220065d30 0x7ff22014dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:30.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff21ed9d700 1 -- 172.21.15.72:0/3554451212 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200b9690 msgr2=0x7ff2200b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:30.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff21ed9d700 1 --2- 172.21.15.72:0/3554451212 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200b9690 0x7ff2200b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:30.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff21ed9d700 1 -- 172.21.15.72:0/3554451212 >> v1:172.21.15.72:6789/0 conn(0x7ff2200bca50 legacy=0x7ff220065620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:30.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff21ed9d700 1 -- 172.21.15.72:0/3554451212 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2200bbbe0 con 0x7ff220065d30 2023-10-18T18:01:30.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff21ed9d700 1 --2- 172.21.15.72:0/3554451212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 0x7ff22014dd50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff218009e60 tx=0x7ff218009780).ready entity=mon.1 client_cookie=fe7c5a33b21e8635 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:30.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff21d59a700 1 -- 172.21.15.72:0/3554451212 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21800e070 con 0x7ff220065d30 2023-10-18T18:01:30.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff21d59a700 1 -- 172.21.15.72:0/3554451212 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21800bb60 con 0x7ff220065d30 2023-10-18T18:01:30.743 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff21d59a700 1 -- 172.21.15.72:0/3554451212 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff218012410 con 0x7ff220065d30 2023-10-18T18:01:30.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff224d69700 1 -- 172.21.15.72:0/3554451212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 msgr2=0x7ff22014dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:30.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff224d69700 1 --2- 172.21.15.72:0/3554451212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 0x7ff22014dd50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff218009e60 tx=0x7ff218009780).stop 2023-10-18T18:01:30.744 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff224d69700 1 -- 172.21.15.72:0/3554451212 shutdown_connections 2023-10-18T18:01:30.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff224d69700 1 --2- 172.21.15.72:0/3554451212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 0x7ff22014dd50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:30.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff224d69700 1 --2- 172.21.15.72:0/3554451212 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200b9690 0x7ff2200b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:30.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff224d69700 1 -- 172.21.15.72:0/3554451212 >> 172.21.15.72:0/3554451212 conn(0x7ff2200620d0 msgr2=0x7ff2200624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:30.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff224d69700 1 -- 172.21.15.72:0/3554451212 shutdown_connections 2023-10-18T18:01:30.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.734+0000 7ff224d69700 1 -- 172.21.15.72:0/3554451212 wait complete. 2023-10-18T18:01:30.746 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff224d69700 1 Processor -- start 2023-10-18T18:01:30.747 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff224d69700 1 -- start start 2023-10-18T18:01:30.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff224d69700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 0x7ff2201611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:30.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff224d69700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2200b9690 0x7ff2201466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:30.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff224d69700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200aeef0 0x7ff22015e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:30.750 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff224d69700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2201620b0 con 0x7ff2200b9690 2023-10-18T18:01:30.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff224d69700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22015e110 con 0x7ff220065d30 2023-10-18T18:01:30.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff224d69700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff220161810 con 0x7ff2200aeef0 2023-10-18T18:01:30.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff21dd9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2200b9690 0x7ff2201466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:30.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff21e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 0x7ff2201611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:30.751 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff21dd9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2200b9690 0x7ff2201466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:51994/0 (socket says 172.21.15.72:51994) 2023-10-18T18:01:30.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff21ed9d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200aeef0 0x7ff22015e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:30.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff21dd9b700 1 -- 172.21.15.72:0/2582397393 learned_addr learned my addr 172.21.15.72:0/2582397393 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:30.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.738+0000 7ff21e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 0x7ff2201611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:43318/0 (socket says 172.21.15.72:43318) 2023-10-18T18:01:30.752 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff21dd9b700 1 -- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200aeef0 msgr2=0x7ff22015e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:30.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff21dd9b700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200aeef0 0x7ff22015e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:30.753 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff21dd9b700 1 -- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 msgr2=0x7ff2201611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:30.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff21dd9b700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 0x7ff2201611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:30.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff21dd9b700 1 -- 172.21.15.72:0/2582397393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff218009a20 con 0x7ff2200b9690 2023-10-18T18:01:30.754 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff21dd9b700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2200b9690 0x7ff2201466f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7ff21400c410 tx=0x7ff214007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:30.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff20e7fc700 1 -- 172.21.15.72:0/2582397393 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140070d0 con 0x7ff2200b9690 2023-10-18T18:01:30.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff20e7fc700 1 -- 172.21.15.72:0/2582397393 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff214007230 con 0x7ff2200b9690 2023-10-18T18:01:30.755 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff220161af0 con 0x7ff2200b9690 2023-10-18T18:01:30.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff20e7fc700 1 -- 172.21.15.72:0/2582397393 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140075b0 con 0x7ff2200b9690 2023-10-18T18:01:30.756 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22015dda0 con 0x7ff2200b9690 2023-10-18T18:01:30.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.742+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22015dc00 con 0x7ff2200b9690 2023-10-18T18:01:30.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.746+0000 7ff20e7fc700 1 -- 172.21.15.72:0/2582397393 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff2140059b0 con 0x7ff2200b9690 2023-10-18T18:01:30.757 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.746+0000 7ff20e7fc700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff20005a510 0x7ff20005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:30.758 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.746+0000 7ff20e7fc700 1 -- 172.21.15.72:0/2582397393 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7ff214074030 con 0x7ff2200b9690 2023-10-18T18:01:30.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.746+0000 7ff21e59c700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff20005a510 0x7ff20005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:30.759 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.746+0000 7ff21e59c700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff20005a510 0x7ff20005c9d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7ff210009c30 tx=0x7ff210009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:30.760 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.750+0000 7ff20e7fc700 1 -- 172.21.15.72:0/2582397393 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ff22015dc00 con 0x7ff2200b9690 2023-10-18T18:01:30.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.946+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff22015dc00 con 0x7ff2200b9690 2023-10-18T18:01:30.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.946+0000 7ff20e7fc700 1 -- 172.21.15.72:0/2582397393 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7ff22015dc00 con 0x7ff2200b9690 2023-10-18T18:01:30.952 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:30.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff20005a510 msgr2=0x7ff20005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:30.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff20005a510 0x7ff20005c9d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7ff210009c30 tx=0x7ff210009060).stop 2023-10-18T18:01:30.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2200b9690 msgr2=0x7ff2201466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:30.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2200b9690 0x7ff2201466f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7ff21400c410 tx=0x7ff214007df0).stop 2023-10-18T18:01:30.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 shutdown_connections 2023-10-18T18:01:30.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff20005a510 0x7ff20005c9d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:30.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff2200aeef0 0x7ff22015e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:30.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2200b9690 0x7ff2201466f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:30.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 --2- 172.21.15.72:0/2582397393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff220065d30 0x7ff2201611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:30.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 >> 172.21.15.72:0/2582397393 conn(0x7ff2200620d0 msgr2=0x7ff22014f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:30.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 shutdown_connections 2023-10-18T18:01:30.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:30.954+0000 7ff224d69700 1 -- 172.21.15.72:0/2582397393 wait complete. 2023-10-18T18:01:30.979 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:33.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~182s 2023-10-18T18:01:33.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~128s 2023-10-18T18:01:33.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~82s 2023-10-18T18:01:33.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~108s 2023-10-18T18:01:33.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~95s 2023-10-18T18:01:33.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~141s 2023-10-18T18:01:33.980 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:34.149 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d36db700 1 Processor -- start 2023-10-18T18:01:34.150 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d36db700 1 -- start start 2023-10-18T18:01:34.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d36db700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f38cc0c2af0 0x7f38cc0c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:34.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d36db700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc0bebd0 con 0x7f38cc14d610 2023-10-18T18:01:34.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d36db700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc0beeb0 con 0x7f38cc0ba9c0 2023-10-18T18:01:34.151 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d36db700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0bf190 con 0x7f38cc0c2af0 2023-10-18T18:01:34.152 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d1477700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f38cc0c2af0 0x7f38cc0c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:34.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d0c76700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f38cc14d610 0x7f38cc149a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:57084/0 (socket says 172.21.15.72:57084) 2023-10-18T18:01:34.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d0c76700 1 -- 172.21.15.72:0/1177186014 learned_addr learned my addr 172.21.15.72:0/1177186014 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:01:34.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d1477700 1 -- 172.21.15.72:0/1177186014 >> v1:172.21.15.179:6789/0 conn(0x7f38cc0ba9c0 legacy=0x7f38cc0bd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:34.153 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d1477700 1 -- 172.21.15.72:0/1177186014 >> v1:172.21.15.72:6789/0 conn(0x7f38cc14d610 legacy=0x7f38cc149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:34.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d1477700 1 -- 172.21.15.72:0/1177186014 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc0bf470 con 0x7f38cc0c2af0 2023-10-18T18:01:34.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.146+0000 7f38d1477700 1 --2- 172.21.15.72:0/1177186014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38cc0c2af0 0x7f38cc0c2ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f38c8006fd0 tx=0x7f38c8006f40).ready entity=mon.0 client_cookie=96237bd50ea2301e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:34.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38c3fff700 1 -- 172.21.15.72:0/1177186014 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8005880 con 0x7f38cc0c2af0 2023-10-18T18:01:34.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38c3fff700 1 -- 172.21.15.72:0/1177186014 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c80059e0 con 0x7f38cc0c2af0 2023-10-18T18:01:34.154 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38c3fff700 1 -- 172.21.15.72:0/1177186014 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8005d60 con 0x7f38cc0c2af0 2023-10-18T18:01:34.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38d36db700 1 -- 172.21.15.72:0/1177186014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38cc0c2af0 msgr2=0x7f38cc0c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:34.155 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38d36db700 1 --2- 172.21.15.72:0/1177186014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38cc0c2af0 0x7f38cc0c2ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f38c8006fd0 tx=0x7f38c8006f40).stop 2023-10-18T18:01:34.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38d36db700 1 -- 172.21.15.72:0/1177186014 shutdown_connections 2023-10-18T18:01:34.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38d36db700 1 --2- 172.21.15.72:0/1177186014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38cc0c2af0 0x7f38cc0c2ed0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:34.156 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38d36db700 1 -- 172.21.15.72:0/1177186014 >> 172.21.15.72:0/1177186014 conn(0x7f38cc064b40 msgr2=0x7f38cc05dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:34.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38d36db700 1 -- 172.21.15.72:0/1177186014 shutdown_connections 2023-10-18T18:01:34.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38d36db700 1 -- 172.21.15.72:0/1177186014 wait complete. 2023-10-18T18:01:34.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38d36db700 1 Processor -- start 2023-10-18T18:01:34.158 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.150+0000 7f38d36db700 1 -- start start 2023-10-18T18:01:34.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d36db700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38cc0ba9c0 0x7f38cc1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:34.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d36db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38cc0c2af0 0x7f38cc164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:34.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d36db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc1421e0 0x7f38cc068560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:34.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d36db700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc161190 con 0x7f38cc0ba9c0 2023-10-18T18:01:34.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d36db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc164930 con 0x7f38cc1421e0 2023-10-18T18:01:34.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d36db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc153450 con 0x7f38cc0c2af0 2023-10-18T18:01:34.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d0c76700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38cc0c2af0 0x7f38cc164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:34.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d0c76700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38cc0c2af0 0x7f38cc164440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:57000/0 (socket says 172.21.15.72:57000) 2023-10-18T18:01:34.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d0c76700 1 -- 172.21.15.72:0/2368072405 learned_addr learned my addr 172.21.15.72:0/2368072405 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:34.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d1c78700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc1421e0 0x7f38cc068560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:34.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d1477700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38cc0ba9c0 0x7f38cc1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:34.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d0c76700 1 -- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc1421e0 msgr2=0x7f38cc068560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:34.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d0c76700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc1421e0 0x7f38cc068560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:34.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d0c76700 1 -- 172.21.15.72:0/2368072405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38cc0ba9c0 msgr2=0x7f38cc1482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:34.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d0c76700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38cc0ba9c0 0x7f38cc1482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:34.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d0c76700 1 -- 172.21.15.72:0/2368072405 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8009250 con 0x7f38cc0c2af0 2023-10-18T18:01:34.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d0c76700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38cc0c2af0 0x7f38cc164440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f38bc00c410 tx=0x7f38bc007e50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:34.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38c17fa700 1 -- 172.21.15.72:0/2368072405 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0070d0 con 0x7f38cc0c2af0 2023-10-18T18:01:34.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38c17fa700 1 -- 172.21.15.72:0/2368072405 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc007230 con 0x7f38cc0c2af0 2023-10-18T18:01:34.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38c17fa700 1 -- 172.21.15.72:0/2368072405 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0075d0 con 0x7f38cc0c2af0 2023-10-18T18:01:34.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc148900 con 0x7f38cc0c2af0 2023-10-18T18:01:34.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.154+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc160860 con 0x7f38cc0c2af0 2023-10-18T18:01:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.158+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc161340 con 0x7f38cc0c2af0 2023-10-18T18:01:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.158+0000 7f38c17fa700 1 -- 172.21.15.72:0/2368072405 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f38bc0059b0 con 0x7f38cc0c2af0 2023-10-18T18:01:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.158+0000 7f38c17fa700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f38b005a510 0x7f38b005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:34.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.162+0000 7f38c17fa700 1 -- 172.21.15.72:0/2368072405 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f38bc0740d0 con 0x7f38cc0c2af0 2023-10-18T18:01:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.162+0000 7f38d1477700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f38b005a510 0x7f38b005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:34.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.162+0000 7f38d1477700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f38b005a510 0x7f38b005c9d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f38c8006fa0 tx=0x7f38c8018000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:34.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.170+0000 7f38c17fa700 1 -- 172.21.15.72:0/2368072405 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f38cc161340 con 0x7f38cc0c2af0 2023-10-18T18:01:34.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.362+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f38cc161340 con 0x7f38cc0c2af0 2023-10-18T18:01:34.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.366+0000 7f38c17fa700 1 -- 172.21.15.72:0/2368072405 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f38cc161340 con 0x7f38cc0c2af0 2023-10-18T18:01:34.369 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:34.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f38b005a510 msgr2=0x7f38b005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:34.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f38b005a510 0x7f38b005c9d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f38c8006fa0 tx=0x7f38c8018000).stop 2023-10-18T18:01:34.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38cc0c2af0 msgr2=0x7f38cc164440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:34.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38cc0c2af0 0x7f38cc164440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f38bc00c410 tx=0x7f38bc007e50).stop 2023-10-18T18:01:34.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 shutdown_connections 2023-10-18T18:01:34.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f38b005a510 0x7f38b005c9d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:34.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38cc1421e0 0x7f38cc068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:34.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f38cc0c2af0 0x7f38cc164440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:34.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 --2- 172.21.15.72:0/2368072405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f38cc0ba9c0 0x7f38cc1482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:34.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 >> 172.21.15.72:0/2368072405 conn(0x7f38cc064b40 msgr2=0x7f38cc0658a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:34.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 shutdown_connections 2023-10-18T18:01:34.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:34.370+0000 7f38d36db700 1 -- 172.21.15.72:0/2368072405 wait complete. 2023-10-18T18:01:34.394 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:37.395 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:37.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47f59e700 1 Processor -- start 2023-10-18T18:01:37.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47f59e700 1 -- start start 2023-10-18T18:01:37.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47f59e700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa4800bdf20 0x7fa4800be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:37.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47f59e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800c2630 0x7fa4800bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:37.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47f59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fa4800686b0 0x7fa480068b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:37.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47f59e700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa480069e70 con 0x7fa4800bdf20 2023-10-18T18:01:37.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47f59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa480069030 con 0x7fa4800686b0 2023-10-18T18:01:37.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47f59e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800bd1a0 con 0x7fa4800c2630 2023-10-18T18:01:37.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47e59c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa4800bdf20 0x7fa4800be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:37.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47e59c700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa4800bdf20 0x7fa4800be300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:52020/0 (socket says 172.21.15.72:52020) 2023-10-18T18:01:37.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47e59c700 1 -- 172.21.15.72:0/1605721305 learned_addr learned my addr 172.21.15.72:0/1605721305 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:37.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47ed9d700 1 --2- 172.21.15.72:0/1605721305 >> v2:172.21.15.179:3300/0 conn(0x7fa4800686b0 0x7fa480068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47dd9b700 1 --2- 172.21.15.72:0/1605721305 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800c2630 0x7fa4800bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47e59c700 1 -- 172.21.15.72:0/1605721305 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800c2630 msgr2=0x7fa4800bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47e59c700 1 --2- 172.21.15.72:0/1605721305 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800c2630 0x7fa4800bc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47e59c700 1 -- 172.21.15.72:0/1605721305 >> v2:172.21.15.179:3300/0 conn(0x7fa4800686b0 msgr2=0x7fa480068b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:37.562 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47e59c700 1 --2- 172.21.15.72:0/1605721305 >> v2:172.21.15.179:3300/0 conn(0x7fa4800686b0 0x7fa480068b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47e59c700 1 -- 172.21.15.72:0/1605721305 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800bd480 con 0x7fa4800bdf20 2023-10-18T18:01:37.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.554+0000 7fa47e59c700 1 --2- 172.21.15.72:0/1605721305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 0x7fa4800be300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fa480067780 tx=0x7fa4700090f0).ready entity=mon.0 client_cookie=9308f3a29544f6d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:37.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47d59a700 1 -- 172.21.15.72:0/1605721305 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47000f510 con 0x7fa4800bdf20 2023-10-18T18:01:37.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47d59a700 1 -- 172.21.15.72:0/1605721305 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47000f670 con 0x7fa4800bdf20 2023-10-18T18:01:37.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47d59a700 1 -- 172.21.15.72:0/1605721305 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47000f9f0 con 0x7fa4800bdf20 2023-10-18T18:01:37.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47f59e700 1 -- 172.21.15.72:0/1605721305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 msgr2=0x7fa4800be300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:37.564 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47f59e700 1 --2- 172.21.15.72:0/1605721305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 0x7fa4800be300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fa480067780 tx=0x7fa4700090f0).stop 2023-10-18T18:01:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47f59e700 1 -- 172.21.15.72:0/1605721305 shutdown_connections 2023-10-18T18:01:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47f59e700 1 --2- 172.21.15.72:0/1605721305 >> v2:172.21.15.179:3300/0 conn(0x7fa4800686b0 0x7fa480068b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47f59e700 1 --2- 172.21.15.72:0/1605721305 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800c2630 0x7fa4800bc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47f59e700 1 --2- 172.21.15.72:0/1605721305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 0x7fa4800be300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47f59e700 1 -- 172.21.15.72:0/1605721305 >> 172.21.15.72:0/1605721305 conn(0x7fa480062ba0 msgr2=0x7fa4800c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47f59e700 1 -- 172.21.15.72:0/1605721305 shutdown_connections 2023-10-18T18:01:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.558+0000 7fa47f59e700 1 -- 172.21.15.72:0/1605721305 wait complete. 2023-10-18T18:01:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47f59e700 1 Processor -- start 2023-10-18T18:01:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47f59e700 1 -- start start 2023-10-18T18:01:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47f59e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800686b0 0x7fa480153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:37.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47f59e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 0x7fa4800a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:37.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47f59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4800c2630 0x7fa4801509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:37.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47f59e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48014f700 con 0x7fa4800bdf20 2023-10-18T18:01:37.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47f59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa480154650 con 0x7fa4800c2630 2023-10-18T18:01:37.568 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47f59e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa480153db0 con 0x7fa4800686b0 2023-10-18T18:01:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47e59c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800686b0 0x7fa480153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47dd9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 0x7fa4800a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4800c2630 0x7fa4801509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47e59c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800686b0 0x7fa480153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:57006/0 (socket says 172.21.15.72:57006) 2023-10-18T18:01:37.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47e59c700 1 -- 172.21.15.72:0/558285324 learned_addr learned my addr 172.21.15.72:0/558285324 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47ed9d700 1 -- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800686b0 msgr2=0x7fa480153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47ed9d700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800686b0 0x7fa480153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47ed9d700 1 -- 172.21.15.72:0/558285324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 msgr2=0x7fa4800a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47ed9d700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 0x7fa4800a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47ed9d700 1 -- 172.21.15.72:0/558285324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470009370 con 0x7fa4800c2630 2023-10-18T18:01:37.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47dd9b700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 0x7fa4800a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:37.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.562+0000 7fa47e59c700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800686b0 0x7fa480153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:37.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.566+0000 7fa47ed9d700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4800c2630 0x7fa4801509e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fa47800ecf0 tx=0x7fa47800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:37.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.566+0000 7fa46e7fc700 1 -- 172.21.15.72:0/558285324 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800c8f0 con 0x7fa4800c2630 2023-10-18T18:01:37.572 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.566+0000 7fa46e7fc700 1 -- 172.21.15.72:0/558285324 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47800f040 con 0x7fa4800c2630 2023-10-18T18:01:37.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.566+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa480154090 con 0x7fa4800c2630 2023-10-18T18:01:37.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.566+0000 7fa46e7fc700 1 -- 172.21.15.72:0/558285324 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800cc60 con 0x7fa4800c2630 2023-10-18T18:01:37.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.566+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4801501e0 con 0x7fa4800c2630 2023-10-18T18:01:37.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.566+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa480046f00 con 0x7fa4800c2630 2023-10-18T18:01:37.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.566+0000 7fa46e7fc700 1 -- 172.21.15.72:0/558285324 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa47801d460 con 0x7fa4800c2630 2023-10-18T18:01:37.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.570+0000 7fa46e7fc700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa46005a540 0x7fa46005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:37.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.570+0000 7fa46e7fc700 1 -- 172.21.15.72:0/558285324 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fa478074fb0 con 0x7fa4800c2630 2023-10-18T18:01:37.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.570+0000 7fa47e59c700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa46005a540 0x7fa46005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:37.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.570+0000 7fa47e59c700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa46005a540 0x7fa46005ca00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fa470009130 tx=0x7fa470017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:37.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.578+0000 7fa46e7fc700 1 -- 172.21.15.72:0/558285324 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fa4780484a0 con 0x7fa4800c2630 2023-10-18T18:01:37.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.770+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa4800c2dc0 con 0x7fa4800c2630 2023-10-18T18:01:37.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.770+0000 7fa46e7fc700 1 -- 172.21.15.72:0/558285324 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fa47804bac0 con 0x7fa4800c2630 2023-10-18T18:01:37.776 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:37.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.774+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa46005a540 msgr2=0x7fa46005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:37.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.774+0000 7fa47f59e700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa46005a540 0x7fa46005ca00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fa470009130 tx=0x7fa470017000).stop 2023-10-18T18:01:37.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.774+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4800c2630 msgr2=0x7fa4801509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:37.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.774+0000 7fa47f59e700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4800c2630 0x7fa4801509e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fa47800ecf0 tx=0x7fa47800b040).stop 2023-10-18T18:01:37.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.774+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 shutdown_connections 2023-10-18T18:01:37.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.774+0000 7fa47f59e700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa46005a540 0x7fa46005ca00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.774+0000 7fa47f59e700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4800c2630 0x7fa4801509e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.778+0000 7fa47f59e700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4800bdf20 0x7fa4800a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.778+0000 7fa47f59e700 1 --2- 172.21.15.72:0/558285324 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4800686b0 0x7fa480153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:37.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.778+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 >> 172.21.15.72:0/558285324 conn(0x7fa480062ba0 msgr2=0x7fa4800c1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:37.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.778+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 shutdown_connections 2023-10-18T18:01:37.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:37.778+0000 7fa47f59e700 1 -- 172.21.15.72:0/558285324 wait complete. 2023-10-18T18:01:37.801 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:39.492 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~188s 2023-10-18T18:01:39.492 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~134s 2023-10-18T18:01:39.492 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~88s 2023-10-18T18:01:39.492 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~114s 2023-10-18T18:01:39.493 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~101s 2023-10-18T18:01:39.493 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~147s 2023-10-18T18:01:40.803 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:40.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.962+0000 7fef5c89b700 1 Processor -- start 2023-10-18T18:01:40.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.962+0000 7fef5c89b700 1 -- start start 2023-10-18T18:01:40.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5c89b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fef58067ff0 0x7fef580683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:40.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5c89b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef58065cd0 con 0x7fef58151410 2023-10-18T18:01:40.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5c89b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef58154750 con 0x7fef580666b0 2023-10-18T18:01:40.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5c89b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef58066160 con 0x7fef58067ff0 2023-10-18T18:01:40.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fef58067ff0 0x7fef580683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:40.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5659c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fef58067ff0 0x7fef580683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:39600/0 (socket says 172.21.15.72:39600) 2023-10-18T18:01:40.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5659c700 1 -- 172.21.15.72:0/2807855860 learned_addr learned my addr 172.21.15.72:0/2807855860 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:40.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5659c700 1 -- 172.21.15.72:0/2807855860 >> v1:172.21.15.179:6789/0 conn(0x7fef580666b0 legacy=0x7fef580bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:40.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5659c700 1 -- 172.21.15.72:0/2807855860 >> v1:172.21.15.72:6789/0 conn(0x7fef58151410 legacy=0x7fef58151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:40.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5659c700 1 -- 172.21.15.72:0/2807855860 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58065100 con 0x7fef58067ff0 2023-10-18T18:01:40.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef5659c700 1 --2- 172.21.15.72:0/2807855860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef58067ff0 0x7fef580683d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fef48006fd0 tx=0x7fef480090f0).ready entity=mon.1 client_cookie=e3dd28550994e370 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:40.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef55d9b700 1 -- 172.21.15.72:0/2807855860 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800b600 con 0x7fef58067ff0 2023-10-18T18:01:40.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef55d9b700 1 -- 172.21.15.72:0/2807855860 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4800b760 con 0x7fef58067ff0 2023-10-18T18:01:40.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.966+0000 7fef55d9b700 1 -- 172.21.15.72:0/2807855860 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800bae0 con 0x7fef58067ff0 2023-10-18T18:01:40.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.970+0000 7fef5c89b700 1 -- 172.21.15.72:0/2807855860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef58067ff0 msgr2=0x7fef580683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:40.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.970+0000 7fef5c89b700 1 --2- 172.21.15.72:0/2807855860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef58067ff0 0x7fef580683d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fef48006fd0 tx=0x7fef480090f0).stop 2023-10-18T18:01:40.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.970+0000 7fef5c89b700 1 -- 172.21.15.72:0/2807855860 shutdown_connections 2023-10-18T18:01:40.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.970+0000 7fef5c89b700 1 --2- 172.21.15.72:0/2807855860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef58067ff0 0x7fef580683d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:40.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.970+0000 7fef5c89b700 1 -- 172.21.15.72:0/2807855860 >> 172.21.15.72:0/2807855860 conn(0x7fef580620d0 msgr2=0x7fef580624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:40.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.970+0000 7fef5c89b700 1 -- 172.21.15.72:0/2807855860 shutdown_connections 2023-10-18T18:01:40.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.970+0000 7fef5c89b700 1 -- 172.21.15.72:0/2807855860 wait complete. 2023-10-18T18:01:40.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.970+0000 7fef5c89b700 1 Processor -- start 2023-10-18T18:01:40.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.970+0000 7fef5c89b700 1 -- start start 2023-10-18T18:01:40.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5c89b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef580666b0 0x7fef580b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:40.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5c89b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef58067ff0 0x7fef580b8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:40.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5c89b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef58151410 0x7fef580b8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:40.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5c89b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef580b7320 con 0x7fef58067ff0 2023-10-18T18:01:40.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5c89b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef580b9400 con 0x7fef580666b0 2023-10-18T18:01:40.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5c89b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef580b9650 con 0x7fef58151410 2023-10-18T18:01:40.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4dd9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef58067ff0 0x7fef580b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:40.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef580666b0 0x7fef580b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:40.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef56d9d700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef58151410 0x7fef580b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:40.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef580666b0 0x7fef580b7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:39612/0 (socket says 172.21.15.72:39612) 2023-10-18T18:01:40.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5659c700 1 -- 172.21.15.72:0/3185308504 learned_addr learned my addr 172.21.15.72:0/3185308504 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:40.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4dd9b700 1 -- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef58151410 msgr2=0x7fef580b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:40.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4dd9b700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef58151410 0x7fef580b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:40.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4dd9b700 1 -- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef580666b0 msgr2=0x7fef580b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:40.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4dd9b700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef580666b0 0x7fef580b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:40.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4dd9b700 1 -- 172.21.15.72:0/3185308504 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef48009370 con 0x7fef58067ff0 2023-10-18T18:01:40.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5659c700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef580666b0 0x7fef580b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:40.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4dd9b700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef58067ff0 0x7fef580b8270 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fef4000c440 tx=0x7fef40007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:40.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4effd700 1 -- 172.21.15.72:0/3185308504 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40007020 con 0x7fef58067ff0 2023-10-18T18:01:40.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4effd700 1 -- 172.21.15.72:0/3185308504 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef40007180 con 0x7fef58067ff0 2023-10-18T18:01:40.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef4effd700 1 -- 172.21.15.72:0/3185308504 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40007520 con 0x7fef58067ff0 2023-10-18T18:01:40.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef580b5330 con 0x7fef58067ff0 2023-10-18T18:01:40.983 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.974+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef580b58f0 con 0x7fef58067ff0 2023-10-18T18:01:40.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.978+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef580b7490 con 0x7fef58067ff0 2023-10-18T18:01:40.984 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.978+0000 7fef4effd700 1 -- 172.21.15.72:0/3185308504 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fef400068b0 con 0x7fef58067ff0 2023-10-18T18:01:40.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.978+0000 7fef4effd700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fef3805a510 0x7fef3805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:40.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.978+0000 7fef4effd700 1 -- 172.21.15.72:0/3185308504 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fef40074320 con 0x7fef58067ff0 2023-10-18T18:01:40.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.982+0000 7fef5659c700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fef3805a510 0x7fef3805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:40.985 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.982+0000 7fef5659c700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fef3805a510 0x7fef3805c9d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fef4800c010 tx=0x7fef48017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:40.990 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:40.986+0000 7fef4effd700 1 -- 172.21.15.72:0/3185308504 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fef580b7490 con 0x7fef58067ff0 2023-10-18T18:01:41.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.178+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fef580b7460 con 0x7fef58067ff0 2023-10-18T18:01:41.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.182+0000 7fef4effd700 1 -- 172.21.15.72:0/3185308504 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fef40047870 con 0x7fef58067ff0 2023-10-18T18:01:41.185 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:41.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.182+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fef3805a510 msgr2=0x7fef3805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:41.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.182+0000 7fef5c89b700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fef3805a510 0x7fef3805c9d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fef4800c010 tx=0x7fef48017000).stop 2023-10-18T18:01:41.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.182+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef58067ff0 msgr2=0x7fef580b8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:41.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.182+0000 7fef5c89b700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef58067ff0 0x7fef580b8270 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fef4000c440 tx=0x7fef40007d40).stop 2023-10-18T18:01:41.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.186+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 shutdown_connections 2023-10-18T18:01:41.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.186+0000 7fef5c89b700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fef3805a510 0x7fef3805c9d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:41.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.186+0000 7fef5c89b700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fef58151410 0x7fef580b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:41.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.186+0000 7fef5c89b700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef58067ff0 0x7fef580b8270 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:41.190 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.186+0000 7fef5c89b700 1 --2- 172.21.15.72:0/3185308504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef580666b0 0x7fef580b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:41.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.186+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 >> 172.21.15.72:0/3185308504 conn(0x7fef580620d0 msgr2=0x7fef58062540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:41.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.186+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 shutdown_connections 2023-10-18T18:01:41.191 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:41.186+0000 7fef5c89b700 1 -- 172.21.15.72:0/3185308504 wait complete. 2023-10-18T18:01:41.210 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:44.211 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:44.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.366+0000 7ff09541b700 1 Processor -- start 2023-10-18T18:01:44.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.366+0000 7ff09541b700 1 -- start start 2023-10-18T18:01:44.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff09541b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff09014b1d0 0x7ff09014e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:44.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff09541b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b5b0 0x7ff0900bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:44.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff09541b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0900bf8d0 0x7ff0900bd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:44.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff09541b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09014dd30 con 0x7ff09014b1d0 2023-10-18T18:01:44.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff09541b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff090062430 con 0x7ff0900bf8d0 2023-10-18T18:01:44.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff09541b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff090062710 con 0x7ff09014b5b0 2023-10-18T18:01:44.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08effd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff09014b1d0 0x7ff09014e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:44.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08ffff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ff0900bf8d0 0x7ff0900bd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:44.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08effd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7ff09014b1d0 0x7ff09014e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49928/0 (socket says 172.21.15.72:49928) 2023-10-18T18:01:44.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08effd700 1 -- 172.21.15.72:0/3931655066 learned_addr learned my addr 172.21.15.72:0/3931655066 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:44.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff0867fc700 1 --2- 172.21.15.72:0/3931655066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b5b0 0x7ff0900bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:44.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08effd700 1 -- 172.21.15.72:0/3931655066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b5b0 msgr2=0x7ff0900bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:44.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08effd700 1 --2- 172.21.15.72:0/3931655066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b5b0 0x7ff0900bcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08effd700 1 -- 172.21.15.72:0/3931655066 >> v2:172.21.15.179:3300/0 conn(0x7ff0900bf8d0 msgr2=0x7ff0900bd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:44.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08effd700 1 --2- 172.21.15.72:0/3931655066 >> v2:172.21.15.179:3300/0 conn(0x7ff0900bf8d0 0x7ff0900bd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08effd700 1 -- 172.21.15.72:0/3931655066 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0900629f0 con 0x7ff09014b1d0 2023-10-18T18:01:44.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff0867fc700 1 --2- 172.21.15.72:0/3931655066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b5b0 0x7ff0900bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08ffff700 1 --2- 172.21.15.72:0/3931655066 >> v2:172.21.15.179:3300/0 conn(0x7ff0900bf8d0 0x7ff0900bd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08effd700 1 --2- 172.21.15.72:0/3931655066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff09014b1d0 0x7ff09014e510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff0900c0830 tx=0x7ff0800090f0).ready entity=mon.0 client_cookie=5df1738cb5edf4b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:44.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08e7fc700 1 -- 172.21.15.72:0/3931655066 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000f510 con 0x7ff09014b1d0 2023-10-18T18:01:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08e7fc700 1 -- 172.21.15.72:0/3931655066 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08000f670 con 0x7ff09014b1d0 2023-10-18T18:01:44.380 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.370+0000 7ff08e7fc700 1 -- 172.21.15.72:0/3931655066 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000fa10 con 0x7ff09014b1d0 2023-10-18T18:01:44.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 -- 172.21.15.72:0/3931655066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff09014b1d0 msgr2=0x7ff09014e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:44.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 --2- 172.21.15.72:0/3931655066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff09014b1d0 0x7ff09014e510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff0900c0830 tx=0x7ff0800090f0).stop 2023-10-18T18:01:44.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 -- 172.21.15.72:0/3931655066 shutdown_connections 2023-10-18T18:01:44.384 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 --2- 172.21.15.72:0/3931655066 >> v2:172.21.15.179:3300/0 conn(0x7ff0900bf8d0 0x7ff0900bd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 --2- 172.21.15.72:0/3931655066 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b5b0 0x7ff0900bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 --2- 172.21.15.72:0/3931655066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff09014b1d0 0x7ff09014e510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 -- 172.21.15.72:0/3931655066 >> 172.21.15.72:0/3931655066 conn(0x7ff0900699e0 msgr2=0x7ff090069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 -- 172.21.15.72:0/3931655066 shutdown_connections 2023-10-18T18:01:44.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 -- 172.21.15.72:0/3931655066 wait complete. 2023-10-18T18:01:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 Processor -- start 2023-10-18T18:01:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.374+0000 7ff09541b700 1 -- start start 2023-10-18T18:01:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff09541b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0900bf8d0 0x7ff090151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:44.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff09541b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b1d0 0x7ff090161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff09541b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0900c2850 0x7ff0901502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff09541b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090162450 con 0x7ff0900bf8d0 2023-10-18T18:01:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff09541b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900b8e50 con 0x7ff0900c2850 2023-10-18T18:01:44.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff09541b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900b9c70 con 0x7ff09014b1d0 2023-10-18T18:01:44.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0900bf8d0 0x7ff090151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:44.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0900bf8d0 0x7ff090151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49944/0 (socket says 172.21.15.72:49944) 2023-10-18T18:01:44.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08effd700 1 -- 172.21.15.72:0/3372805178 learned_addr learned my addr 172.21.15.72:0/3372805178 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:44.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08ffff700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0900c2850 0x7ff0901502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:44.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff0867fc700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b1d0 0x7ff090161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:44.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08effd700 1 -- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b1d0 msgr2=0x7ff090161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:44.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08effd700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b1d0 0x7ff090161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08effd700 1 -- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0900c2850 msgr2=0x7ff0901502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:44.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08effd700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0900c2850 0x7ff0901502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08effd700 1 -- 172.21.15.72:0/3372805178 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff088009a20 con 0x7ff0900bf8d0 2023-10-18T18:01:44.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff0867fc700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b1d0 0x7ff090161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:01:44.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff08effd700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0900bf8d0 0x7ff090151530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7ff080006fd0 tx=0x7ff080017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:44.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff0877fe700 1 -- 172.21.15.72:0/3372805178 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080006950 con 0x7ff0900bf8d0 2023-10-18T18:01:44.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff0877fe700 1 -- 172.21.15.72:0/3372805178 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff080000bc0 con 0x7ff0900bf8d0 2023-10-18T18:01:44.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff078009250 con 0x7ff0900bf8d0 2023-10-18T18:01:44.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff0877fe700 1 -- 172.21.15.72:0/3372805178 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080024470 con 0x7ff0900bf8d0 2023-10-18T18:01:44.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.378+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff080009370 con 0x7ff0900bf8d0 2023-10-18T18:01:44.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.382+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09006c190 con 0x7ff0900bf8d0 2023-10-18T18:01:44.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.382+0000 7ff0877fe700 1 -- 172.21.15.72:0/3372805178 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ff080021070 con 0x7ff0900bf8d0 2023-10-18T18:01:44.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.382+0000 7ff0877fe700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff07405a540 0x7ff07405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:44.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.382+0000 7ff0877fe700 1 -- 172.21.15.72:0/3372805178 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7ff080076360 con 0x7ff0900bf8d0 2023-10-18T18:01:44.393 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.382+0000 7ff0867fc700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff07405a540 0x7ff07405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.386+0000 7ff0867fc700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff07405a540 0x7ff07405ca00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ff0900bd150 tx=0x7ff078008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:44.394 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.390+0000 7ff0877fe700 1 -- 172.21.15.72:0/3372805178 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ff0800498b0 con 0x7ff0900bf8d0 2023-10-18T18:01:44.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.578+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff090062cd0 con 0x7ff0900bf8d0 2023-10-18T18:01:44.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.578+0000 7ff0877fe700 1 -- 172.21.15.72:0/3372805178 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7ff08004ced0 con 0x7ff0900bf8d0 2023-10-18T18:01:44.585 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:44.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff07405a540 msgr2=0x7ff07405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:44.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff07405a540 0x7ff07405ca00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ff0900bd150 tx=0x7ff078008040).stop 2023-10-18T18:01:44.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0900bf8d0 msgr2=0x7ff090151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:44.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0900bf8d0 0x7ff090151530 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7ff080006fd0 tx=0x7ff080017000).stop 2023-10-18T18:01:44.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 shutdown_connections 2023-10-18T18:01:44.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ff07405a540 0x7ff07405ca00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0900c2850 0x7ff0901502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ff09014b1d0 0x7ff090161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 --2- 172.21.15.72:0/3372805178 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff0900bf8d0 0x7ff090151530 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:44.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.582+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 >> 172.21.15.72:0/3372805178 conn(0x7ff0900699e0 msgr2=0x7ff090069f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:44.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.586+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 shutdown_connections 2023-10-18T18:01:44.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:44.586+0000 7ff09541b700 1 -- 172.21.15.72:0/3372805178 wait complete. 2023-10-18T18:01:44.609 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:45.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~195s 2023-10-18T18:01:45.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~140s 2023-10-18T18:01:45.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~95s 2023-10-18T18:01:45.803 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~121s 2023-10-18T18:01:45.803 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~107s 2023-10-18T18:01:45.803 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~154s 2023-10-18T18:01:47.611 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:47.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.769+0000 7f003b59e700 1 Processor -- start 2023-10-18T18:01:47.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.769+0000 7f003b59e700 1 -- start start 2023-10-18T18:01:47.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f003c0658a0 0x7f003c065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:47.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0bd1c0 0x7f003c14ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:47.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c0be2e0 con 0x7f003c0c2bf0 2023-10-18T18:01:47.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003c0bda80 con 0x7f003c0658a0 2023-10-18T18:01:47.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c06a7c0 con 0x7f003c0bd1c0 2023-10-18T18:01:47.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003a59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f003c0658a0 0x7f003c065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:47.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003a59c700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f003c0658a0 0x7f003c065c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:39636/0 (socket says 172.21.15.72:39636) 2023-10-18T18:01:47.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003a59c700 1 -- 172.21.15.72:0/3414539959 learned_addr learned my addr 172.21.15.72:0/3414539959 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:47.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003ad9d700 1 --2- 172.21.15.72:0/3414539959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0bd1c0 0x7f003c14ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:47.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f0031d9b700 1 --1- 172.21.15.72:0/3414539959 >> v1:172.21.15.179:6789/0 conn(0x7f003c0c2bf0 0x7f003c0630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:37722/0 (socket says 172.21.15.72:37722) 2023-10-18T18:01:47.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003a59c700 1 -- 172.21.15.72:0/3414539959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0bd1c0 msgr2=0x7f003c14ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:47.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003a59c700 1 --2- 172.21.15.72:0/3414539959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0bd1c0 0x7f003c14ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:47.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003a59c700 1 -- 172.21.15.72:0/3414539959 >> v1:172.21.15.179:6789/0 conn(0x7f003c0c2bf0 legacy=0x7f003c0630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:47.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003a59c700 1 -- 172.21.15.72:0/3414539959 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0be770 con 0x7f003c0658a0 2023-10-18T18:01:47.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003a59c700 1 --2- 172.21.15.72:0/3414539959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0658a0 0x7f003c065c80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f002c009f20 tx=0x7f002c009840).ready entity=mon.1 client_cookie=58dabebb9ed6b816 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:47.781 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f0039d9b700 1 -- 172.21.15.72:0/3414539959 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00e070 con 0x7f003c0658a0 2023-10-18T18:01:47.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f0039d9b700 1 -- 172.21.15.72:0/3414539959 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c00b9d0 con 0x7f003c0658a0 2023-10-18T18:01:47.782 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f0039d9b700 1 -- 172.21.15.72:0/3414539959 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00bd50 con 0x7f003c0658a0 2023-10-18T18:01:47.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 -- 172.21.15.72:0/3414539959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0658a0 msgr2=0x7f003c065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:47.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 --2- 172.21.15.72:0/3414539959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0658a0 0x7f003c065c80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f002c009f20 tx=0x7f002c009840).stop 2023-10-18T18:01:47.783 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 -- 172.21.15.72:0/3414539959 shutdown_connections 2023-10-18T18:01:47.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 --2- 172.21.15.72:0/3414539959 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0bd1c0 0x7f003c14ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:47.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 --2- 172.21.15.72:0/3414539959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0658a0 0x7f003c065c80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:47.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.773+0000 7f003b59e700 1 -- 172.21.15.72:0/3414539959 >> 172.21.15.72:0/3414539959 conn(0x7f003c062ba0 msgr2=0x7f003c0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:47.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 -- 172.21.15.72:0/3414539959 shutdown_connections 2023-10-18T18:01:47.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 -- 172.21.15.72:0/3414539959 wait complete. 2023-10-18T18:01:47.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 Processor -- start 2023-10-18T18:01:47.785 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 -- start start 2023-10-18T18:01:47.786 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0658a0 0x7f003c1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:47.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0bd1c0 0x7f003c1608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:47.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f003c0c2bf0 0x7f003c14eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:47.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c14dae0 con 0x7f003c0c2bf0 2023-10-18T18:01:47.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c14d810 con 0x7f003c0bd1c0 2023-10-18T18:01:47.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003b59e700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0c0b60 con 0x7f003c0658a0 2023-10-18T18:01:47.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f0031d9b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0bd1c0 0x7f003c1608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:47.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003a59c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0658a0 0x7f003c1603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:47.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f003c0c2bf0 0x7f003c14eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:47.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.777+0000 7f003a59c700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0658a0 0x7f003c1603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:43420/0 (socket says 172.21.15.72:43420) 2023-10-18T18:01:47.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003a59c700 1 -- 172.21.15.72:0/562496047 learned_addr learned my addr 172.21.15.72:0/562496047 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:47.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003ad9d700 1 -- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0658a0 msgr2=0x7f003c1603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:47.788 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003ad9d700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0658a0 0x7f003c1603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:47.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003ad9d700 1 -- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0bd1c0 msgr2=0x7f003c1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:47.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003ad9d700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0bd1c0 0x7f003c1608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:47.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003ad9d700 1 -- 172.21.15.72:0/562496047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c009ae0 con 0x7f003c0c2bf0 2023-10-18T18:01:47.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f0031d9b700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0bd1c0 0x7f003c1608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:47.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003a59c700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0658a0 0x7f003c1603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:47.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003ad9d700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f003c0c2bf0 0x7f003c14eeb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f003400e930 tx=0x7f0034009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:47.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f0032ffd700 1 -- 172.21.15.72:0/562496047 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034012600 con 0x7f003c0c2bf0 2023-10-18T18:01:47.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f0032ffd700 1 -- 172.21.15.72:0/562496047 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0034012760 con 0x7f003c0c2bf0 2023-10-18T18:01:47.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f0032ffd700 1 -- 172.21.15.72:0/562496047 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034012ae0 con 0x7f003c0c2bf0 2023-10-18T18:01:47.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c160f80 con 0x7f003c0c2bf0 2023-10-18T18:01:47.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c150e00 con 0x7f003c0c2bf0 2023-10-18T18:01:47.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.781+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c14dc90 con 0x7f003c0c2bf0 2023-10-18T18:01:47.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.785+0000 7f0032ffd700 1 -- 172.21.15.72:0/562496047 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f003401b400 con 0x7f003c0c2bf0 2023-10-18T18:01:47.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.785+0000 7f0032ffd700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f001c05a540 0x7f001c05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:47.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.785+0000 7f0032ffd700 1 -- 172.21.15.72:0/562496047 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f0034074d10 con 0x7f003c0c2bf0 2023-10-18T18:01:47.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.785+0000 7f003a59c700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f001c05a540 0x7f001c05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:47.792 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.785+0000 7f003a59c700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f001c05a540 0x7f001c05ca00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f003c067180 tx=0x7f002c009840).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:47.797 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.793+0000 7f0032ffd700 1 -- 172.21.15.72:0/562496047 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f003c14dc90 con 0x7f003c0c2bf0 2023-10-18T18:01:47.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.981+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f003c14dc90 con 0x7f003c0c2bf0 2023-10-18T18:01:47.988 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.981+0000 7f0032ffd700 1 -- 172.21.15.72:0/562496047 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f003c14dc90 con 0x7f003c0c2bf0 2023-10-18T18:01:47.989 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:47.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f001c05a540 msgr2=0x7f001c05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:47.991 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f001c05a540 0x7f001c05ca00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f003c067180 tx=0x7f002c009840).stop 2023-10-18T18:01:47.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f003c0c2bf0 msgr2=0x7f003c14eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:47.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f003c0c2bf0 0x7f003c14eeb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f003400e930 tx=0x7f0034009060).stop 2023-10-18T18:01:47.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 shutdown_connections 2023-10-18T18:01:47.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f001c05a540 0x7f001c05ca00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:47.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f003c0c2bf0 0x7f003c14eeb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:47.992 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f003c0bd1c0 0x7f003c1608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:47.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 --2- 172.21.15.72:0/562496047 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f003c0658a0 0x7f003c1603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:47.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 >> 172.21.15.72:0/562496047 conn(0x7f003c062ba0 msgr2=0x7f003c153450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:47.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 shutdown_connections 2023-10-18T18:01:47.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:47.985+0000 7f003b59e700 1 -- 172.21.15.72:0/562496047 wait complete. 2023-10-18T18:01:48.010 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:51.011 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:51.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.173+0000 7f98b2c5b700 1 Processor -- start 2023-10-18T18:01:51.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.173+0000 7f98b2c5b700 1 -- start start 2023-10-18T18:01:51.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b2c5b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac149d20 0x7f98ac1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:51.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b2c5b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac151350 con 0x7f98ac14a100 2023-10-18T18:01:51.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b2c5b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac150120 con 0x7f98ac150aa0 2023-10-18T18:01:51.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b2c5b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac14fe80 con 0x7f98ac149d20 2023-10-18T18:01:51.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b09f7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac149d20 0x7f98ac1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:51.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b11f8700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f98ac14a100 0x7f98ac15a7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56848/0 (socket says 172.21.15.72:56848) 2023-10-18T18:01:51.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b11f8700 1 -- 172.21.15.72:0/2527464245 learned_addr learned my addr 172.21.15.72:0/2527464245 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:01:51.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98abfff700 1 --1- 172.21.15.72:0/2527464245 >> v1:172.21.15.179:6789/0 conn(0x7f98ac150aa0 0x7f98ac0b9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:36228/0 (socket says 172.21.15.72:36228) 2023-10-18T18:01:51.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b09f7700 1 -- 172.21.15.72:0/2527464245 >> v1:172.21.15.179:6789/0 conn(0x7f98ac150aa0 legacy=0x7f98ac0b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:51.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b09f7700 1 -- 172.21.15.72:0/2527464245 >> v1:172.21.15.72:6789/0 conn(0x7f98ac14a100 legacy=0x7f98ac15a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:51.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b09f7700 1 -- 172.21.15.72:0/2527464245 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac153890 con 0x7f98ac149d20 2023-10-18T18:01:51.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98b09f7700 1 --2- 172.21.15.72:0/2527464245 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac149d20 0x7f98ac1505b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f98a0009790 tx=0x7f98a0009060).ready entity=mon.2 client_cookie=c342ea3c2f37c21f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:51.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98ab7fe700 1 -- 172.21.15.72:0/2527464245 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a000b5a0 con 0x7f98ac149d20 2023-10-18T18:01:51.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98ab7fe700 1 -- 172.21.15.72:0/2527464245 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a000b700 con 0x7f98ac149d20 2023-10-18T18:01:51.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.177+0000 7f98ab7fe700 1 -- 172.21.15.72:0/2527464245 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a000ba80 con 0x7f98ac149d20 2023-10-18T18:01:51.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.181+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2527464245 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac149d20 msgr2=0x7f98ac1505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:51.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.181+0000 7f98b2c5b700 1 --2- 172.21.15.72:0/2527464245 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac149d20 0x7f98ac1505b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f98a0009790 tx=0x7f98a0009060).stop 2023-10-18T18:01:51.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.181+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2527464245 shutdown_connections 2023-10-18T18:01:51.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.181+0000 7f98b2c5b700 1 --2- 172.21.15.72:0/2527464245 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac149d20 0x7f98ac1505b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:51.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.181+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2527464245 >> 172.21.15.72:0/2527464245 conn(0x7f98ac065ab0 msgr2=0x7f98ac065ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:51.187 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.181+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2527464245 shutdown_connections 2023-10-18T18:01:51.188 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.181+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2527464245 wait complete. 2023-10-18T18:01:51.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b2c5b700 1 Processor -- start 2023-10-18T18:01:51.189 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b2c5b700 1 -- start start 2023-10-18T18:01:51.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b2c5b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98ac149d20 0x7f98ac13ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:51.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b2c5b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac14a100 0x7f98ac147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:51.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b2c5b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac150aa0 0x7f98ac13bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:51.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b2c5b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c3820 con 0x7f98ac149d20 2023-10-18T18:01:51.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b2c5b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac14da50 con 0x7f98ac150aa0 2023-10-18T18:01:51.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b2c5b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac1420e0 con 0x7f98ac14a100 2023-10-18T18:01:51.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b09f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98ac149d20 0x7f98ac13ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:51.193 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b09f7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98ac149d20 0x7f98ac13ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60376/0 (socket says 172.21.15.72:60376) 2023-10-18T18:01:51.194 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.185+0000 7f98b09f7700 1 -- 172.21.15.72:0/2388834036 learned_addr learned my addr 172.21.15.72:0/2388834036 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:51.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98abfff700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac14a100 0x7f98ac147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:51.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b11f8700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac150aa0 0x7f98ac13bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:51.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b09f7700 1 -- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac14a100 msgr2=0x7f98ac147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:51.195 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b09f7700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac14a100 0x7f98ac147e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:51.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b09f7700 1 -- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac150aa0 msgr2=0x7f98ac13bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:51.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b09f7700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac150aa0 0x7f98ac13bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:51.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b09f7700 1 -- 172.21.15.72:0/2388834036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00092a0 con 0x7f98ac149d20 2023-10-18T18:01:51.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b09f7700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98ac149d20 0x7f98ac13ca90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f98ac06a3e0 tx=0x7f98a0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:51.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98a8ff9700 1 -- 172.21.15.72:0/2388834036 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a000b870 con 0x7f98ac149d20 2023-10-18T18:01:51.196 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98a8ff9700 1 -- 172.21.15.72:0/2388834036 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a000b9d0 con 0x7f98ac149d20 2023-10-18T18:01:51.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac0bcd50 con 0x7f98ac149d20 2023-10-18T18:01:51.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98a8ff9700 1 -- 172.21.15.72:0/2388834036 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a001b690 con 0x7f98ac149d20 2023-10-18T18:01:51.197 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac0bc870 con 0x7f98ac149d20 2023-10-18T18:01:51.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.189+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac06c190 con 0x7f98ac149d20 2023-10-18T18:01:51.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.193+0000 7f98a8ff9700 1 -- 172.21.15.72:0/2388834036 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f98a001ba40 con 0x7f98ac149d20 2023-10-18T18:01:51.198 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.193+0000 7f98a8ff9700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989005a540 0x7f989005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:51.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.193+0000 7f98a8ff9700 1 -- 172.21.15.72:0/2388834036 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f98a0076580 con 0x7f98ac149d20 2023-10-18T18:01:51.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.193+0000 7f98abfff700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989005a540 0x7f989005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:51.199 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.193+0000 7f98abfff700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989005a540 0x7f989005ca00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f98ac151930 tx=0x7f9898007bf0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:51.204 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.201+0000 7f98a8ff9700 1 -- 172.21.15.72:0/2388834036 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f98a0049ad0 con 0x7f98ac149d20 2023-10-18T18:01:51.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.397+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f98ac153b70 con 0x7f98ac149d20 2023-10-18T18:01:51.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.397+0000 7f98a8ff9700 1 -- 172.21.15.72:0/2388834036 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f98a00280e0 con 0x7f98ac149d20 2023-10-18T18:01:51.403 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:51.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.397+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989005a540 msgr2=0x7f989005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:51.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.397+0000 7f98b2c5b700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989005a540 0x7f989005ca00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f98ac151930 tx=0x7f9898007bf0).stop 2023-10-18T18:01:51.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.397+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98ac149d20 msgr2=0x7f98ac13ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:51.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.397+0000 7f98b2c5b700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98ac149d20 0x7f98ac13ca90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f98ac06a3e0 tx=0x7f98a0015000).stop 2023-10-18T18:01:51.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.401+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 shutdown_connections 2023-10-18T18:01:51.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.401+0000 7f98b2c5b700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f989005a540 0x7f989005ca00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:51.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.401+0000 7f98b2c5b700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f98ac150aa0 0x7f98ac13bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:51.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.401+0000 7f98b2c5b700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f98ac14a100 0x7f98ac147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:51.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.401+0000 7f98b2c5b700 1 --2- 172.21.15.72:0/2388834036 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98ac149d20 0x7f98ac13ca90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:51.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.401+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 >> 172.21.15.72:0/2388834036 conn(0x7f98ac065ab0 msgr2=0x7f98ac0c4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:51.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.401+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 shutdown_connections 2023-10-18T18:01:51.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:51.401+0000 7f98b2c5b700 1 -- 172.21.15.72:0/2388834036 wait complete. 2023-10-18T18:01:51.424 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:52.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~201s 2023-10-18T18:01:52.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~147s 2023-10-18T18:01:52.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~101s 2023-10-18T18:01:52.113 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~127s 2023-10-18T18:01:52.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~114s 2023-10-18T18:01:52.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~160s 2023-10-18T18:01:54.425 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.589+0000 7fa5032db700 1 Processor -- start 2023-10-18T18:01:54.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa5032db700 1 -- start start 2023-10-18T18:01:54.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa5032db700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa4fc067ff0 0x7fa4fc0683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa5032db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc151410 0x7fa4fc151860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa5032db700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4fc065cd0 con 0x7fa4fc0666b0 2023-10-18T18:01:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa5032db700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4fc066160 con 0x7fa4fc067ff0 2023-10-18T18:01:54.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa5032db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc154750 con 0x7fa4fc151410 2023-10-18T18:01:54.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa501077700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa4fc067ff0 0x7fa4fc0683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:54.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa501878700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc151410 0x7fa4fc151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:54.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa501077700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fa4fc067ff0 0x7fa4fc0683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60382/0 (socket says 172.21.15.72:60382) 2023-10-18T18:01:54.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa501077700 1 -- 172.21.15.72:0/2490823291 learned_addr learned my addr 172.21.15.72:0/2490823291 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:54.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa500876700 1 --1- 172.21.15.72:0/2490823291 >> v1:172.21.15.72:6789/0 conn(0x7fa4fc0666b0 0x7fa4fc0bd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:56860/0 (socket says 172.21.15.72:56860) 2023-10-18T18:01:54.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa501077700 1 -- 172.21.15.72:0/2490823291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc151410 msgr2=0x7fa4fc151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:54.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa501077700 1 --2- 172.21.15.72:0/2490823291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc151410 0x7fa4fc151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:54.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa501077700 1 -- 172.21.15.72:0/2490823291 >> v1:172.21.15.72:6789/0 conn(0x7fa4fc0666b0 legacy=0x7fa4fc0bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:54.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.593+0000 7fa501077700 1 -- 172.21.15.72:0/2490823291 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc065100 con 0x7fa4fc067ff0 2023-10-18T18:01:54.603 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa501077700 1 --2- 172.21.15.72:0/2490823291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc067ff0 0x7fa4fc0683d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fa4f8006fd0 tx=0x7fa4f80090f0).ready entity=mon.0 client_cookie=87a8f584ac7c1c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:54.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa4f3fff700 1 -- 172.21.15.72:0/2490823291 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800b580 con 0x7fa4fc067ff0 2023-10-18T18:01:54.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa4f3fff700 1 -- 172.21.15.72:0/2490823291 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f800b6e0 con 0x7fa4fc067ff0 2023-10-18T18:01:54.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa4f3fff700 1 -- 172.21.15.72:0/2490823291 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800ba60 con 0x7fa4fc067ff0 2023-10-18T18:01:54.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa5032db700 1 -- 172.21.15.72:0/2490823291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc067ff0 msgr2=0x7fa4fc0683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:54.604 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa5032db700 1 --2- 172.21.15.72:0/2490823291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc067ff0 0x7fa4fc0683d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fa4f8006fd0 tx=0x7fa4f80090f0).stop 2023-10-18T18:01:54.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa5032db700 1 -- 172.21.15.72:0/2490823291 shutdown_connections 2023-10-18T18:01:54.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa5032db700 1 --2- 172.21.15.72:0/2490823291 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc151410 0x7fa4fc151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:54.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa5032db700 1 --2- 172.21.15.72:0/2490823291 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc067ff0 0x7fa4fc0683d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:54.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa5032db700 1 -- 172.21.15.72:0/2490823291 >> 172.21.15.72:0/2490823291 conn(0x7fa4fc0620d0 msgr2=0x7fa4fc0624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:54.605 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa5032db700 1 -- 172.21.15.72:0/2490823291 shutdown_connections 2023-10-18T18:01:54.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.597+0000 7fa5032db700 1 -- 172.21.15.72:0/2490823291 wait complete. 2023-10-18T18:01:54.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa5032db700 1 Processor -- start 2023-10-18T18:01:54.606 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa5032db700 1 -- start start 2023-10-18T18:01:54.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa5032db700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc0666b0 0x7fa4fc0be8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:54.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa5032db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc067ff0 0x7fa4fc0b7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:54.610 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa5032db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4fc151410 0x7fa4fc0b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:54.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa5032db700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc0b8070 con 0x7fa4fc0666b0 2023-10-18T18:01:54.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa5032db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc14f090 con 0x7fa4fc151410 2023-10-18T18:01:54.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa5032db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc164690 con 0x7fa4fc067ff0 2023-10-18T18:01:54.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa501077700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc0666b0 0x7fa4fc0be8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:54.611 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa500876700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc067ff0 0x7fa4fc0b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:54.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa501878700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4fc151410 0x7fa4fc0b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:54.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa501077700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc0666b0 0x7fa4fc0be8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60398/0 (socket says 172.21.15.72:60398) 2023-10-18T18:01:54.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.601+0000 7fa501077700 1 -- 172.21.15.72:0/1703633306 learned_addr learned my addr 172.21.15.72:0/1703633306 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:54.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa501878700 1 -- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc067ff0 msgr2=0x7fa4fc0b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:54.612 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa501878700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc067ff0 0x7fa4fc0b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:54.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa501878700 1 -- 172.21.15.72:0/1703633306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc0666b0 msgr2=0x7fa4fc0be8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:54.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa501878700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc0666b0 0x7fa4fc0be8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:54.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa501878700 1 -- 172.21.15.72:0/1703633306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8009370 con 0x7fa4fc151410 2023-10-18T18:01:54.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa500876700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc067ff0 0x7fa4fc0b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:54.613 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa501077700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc0666b0 0x7fa4fc0be8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:01:54.614 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa501878700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4fc151410 0x7fa4fc0b7b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa4f400ecc0 tx=0x7fa4f400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:54.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa4f17fa700 1 -- 172.21.15.72:0/1703633306 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f400c8c0 con 0x7fa4fc151410 2023-10-18T18:01:54.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa4f17fa700 1 -- 172.21.15.72:0/1703633306 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f400f040 con 0x7fa4fc151410 2023-10-18T18:01:54.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc164950 con 0x7fa4fc151410 2023-10-18T18:01:54.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa4f17fa700 1 -- 172.21.15.72:0/1703633306 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f400cc30 con 0x7fa4fc151410 2023-10-18T18:01:54.615 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.605+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc164f10 con 0x7fa4fc151410 2023-10-18T18:01:54.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.609+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4fc06c190 con 0x7fa4fc151410 2023-10-18T18:01:54.616 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.609+0000 7fa4f17fa700 1 -- 172.21.15.72:0/1703633306 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa4f401d460 con 0x7fa4fc151410 2023-10-18T18:01:54.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.609+0000 7fa4f17fa700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4e005a510 0x7fa4e005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:54.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.609+0000 7fa4f17fa700 1 -- 172.21.15.72:0/1703633306 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fa4f4075a30 con 0x7fa4fc151410 2023-10-18T18:01:54.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.609+0000 7fa501077700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4e005a510 0x7fa4e005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:54.617 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.613+0000 7fa501077700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4e005a510 0x7fa4e005c9d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fa4f800b340 tx=0x7fa4f8017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:54.624 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.617+0000 7fa4f17fa700 1 -- 172.21.15.72:0/1703633306 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fa4f4049080 con 0x7fa4fc151410 2023-10-18T18:01:54.820 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.813+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa4fc0653a0 con 0x7fa4fc151410 2023-10-18T18:01:54.821 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa4f17fa700 1 -- 172.21.15.72:0/1703633306 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fa4f404c6a0 con 0x7fa4fc151410 2023-10-18T18:01:54.821 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:54.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4e005a510 msgr2=0x7fa4e005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:54.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4e005a510 0x7fa4e005c9d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fa4f800b340 tx=0x7fa4f8017000).stop 2023-10-18T18:01:54.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4fc151410 msgr2=0x7fa4fc0b7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:54.824 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4fc151410 0x7fa4fc0b7b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fa4f400ecc0 tx=0x7fa4f400b040).stop 2023-10-18T18:01:54.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 shutdown_connections 2023-10-18T18:01:54.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa4e005a510 0x7fa4e005c9d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:54.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4fc151410 0x7fa4fc0b7b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:54.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa4fc067ff0 0x7fa4fc0b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:54.825 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 --2- 172.21.15.72:0/1703633306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4fc0666b0 0x7fa4fc0be8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:54.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 >> 172.21.15.72:0/1703633306 conn(0x7fa4fc0620d0 msgr2=0x7fa4fc1505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:54.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 shutdown_connections 2023-10-18T18:01:54.826 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:54.817+0000 7fa5032db700 1 -- 172.21.15.72:0/1703633306 wait complete. 2023-10-18T18:01:54.843 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:56.840 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:01:56.848 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:01:57.845 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:01:58.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:57.997+0000 7fbebd879700 1 Processor -- start 2023-10-18T18:01:58.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:57.997+0000 7fbebd879700 1 -- start start 2023-10-18T18:01:58.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fbeb80658a0 0x7fbeb8065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:58.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb80630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:58.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fbeb80635d0 0x7fbeb8151290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:58.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb80bda80 con 0x7fbeb80658a0 2023-10-18T18:01:58.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb806a7c0 con 0x7fbeb80635d0 2023-10-18T18:01:58.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80be2e0 con 0x7fbeb80c2bf0 2023-10-18T18:01:58.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb7fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb80630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:58.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb7fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb80630e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:56898/0 (socket says 172.21.15.72:56898) 2023-10-18T18:01:58.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebc877700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fbeb80658a0 0x7fbeb8065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:58.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb7fff700 1 -- 172.21.15.72:0/3302488958 learned_addr learned my addr 172.21.15.72:0/3302488958 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:58.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd078700 1 --2- 172.21.15.72:0/3302488958 >> v2:172.21.15.179:3300/0 conn(0x7fbeb80635d0 0x7fbeb8151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:58.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb7fff700 1 -- 172.21.15.72:0/3302488958 >> v2:172.21.15.179:3300/0 conn(0x7fbeb80635d0 msgr2=0x7fbeb8151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:58.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb7fff700 1 --2- 172.21.15.72:0/3302488958 >> v2:172.21.15.179:3300/0 conn(0x7fbeb80635d0 0x7fbeb8151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb7fff700 1 -- 172.21.15.72:0/3302488958 >> v2:172.21.15.72:3300/0 conn(0x7fbeb80658a0 msgr2=0x7fbeb8065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:01:58.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb7fff700 1 --2- 172.21.15.72:0/3302488958 >> v2:172.21.15.72:3300/0 conn(0x7fbeb80658a0 0x7fbeb8065c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb7fff700 1 -- 172.21.15.72:0/3302488958 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb80be770 con 0x7fbeb80c2bf0 2023-10-18T18:01:58.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd078700 1 --2- 172.21.15.72:0/3302488958 >> v2:172.21.15.179:3300/0 conn(0x7fbeb80635d0 0x7fbeb8151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:01:58.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb7fff700 1 --2- 172.21.15.72:0/3302488958 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb80630e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fbeb80c3010 tx=0x7fbea40090f0).ready entity=mon.2 client_cookie=a00310185221a042 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:58.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb77fe700 1 -- 172.21.15.72:0/3302488958 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400b520 con 0x7fbeb80c2bf0 2023-10-18T18:01:58.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb77fe700 1 -- 172.21.15.72:0/3302488958 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea400b680 con 0x7fbeb80c2bf0 2023-10-18T18:01:58.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbeb77fe700 1 -- 172.21.15.72:0/3302488958 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400ba00 con 0x7fbeb80c2bf0 2023-10-18T18:01:58.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 -- 172.21.15.72:0/3302488958 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 msgr2=0x7fbeb80630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 --2- 172.21.15.72:0/3302488958 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb80630e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fbeb80c3010 tx=0x7fbea40090f0).stop 2023-10-18T18:01:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 -- 172.21.15.72:0/3302488958 shutdown_connections 2023-10-18T18:01:58.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 --2- 172.21.15.72:0/3302488958 >> v2:172.21.15.179:3300/0 conn(0x7fbeb80635d0 0x7fbeb8151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 --2- 172.21.15.72:0/3302488958 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb80630e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 --2- 172.21.15.72:0/3302488958 >> v2:172.21.15.72:3300/0 conn(0x7fbeb80658a0 0x7fbeb8065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.012 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 -- 172.21.15.72:0/3302488958 >> 172.21.15.72:0/3302488958 conn(0x7fbeb8062ba0 msgr2=0x7fbeb80c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:58.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 -- 172.21.15.72:0/3302488958 shutdown_connections 2023-10-18T18:01:58.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.001+0000 7fbebd879700 1 -- 172.21.15.72:0/3302488958 wait complete. 2023-10-18T18:01:58.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.005+0000 7fbebd879700 1 Processor -- start 2023-10-18T18:01:58.013 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.005+0000 7fbebd879700 1 -- start start 2023-10-18T18:01:58.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.005+0000 7fbebd879700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb80635d0 0x7fbeb80bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:58.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.005+0000 7fbebd879700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbeb80658a0 0x7fbeb81600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:58.015 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.005+0000 7fbebd879700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb81605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:58.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.005+0000 7fbebd879700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81633f0 con 0x7fbeb80658a0 2023-10-18T18:01:58.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.005+0000 7fbebd879700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8160ab0 con 0x7fbeb80635d0 2023-10-18T18:01:58.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.005+0000 7fbebd879700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb80698e0 con 0x7fbeb80c2bf0 2023-10-18T18:01:58.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb7fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbeb80658a0 0x7fbeb81600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:58.016 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb7fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbeb80658a0 0x7fbeb81600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60408/0 (socket says 172.21.15.72:60408) 2023-10-18T18:01:58.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb7fff700 1 -- 172.21.15.72:0/1238357292 learned_addr learned my addr 172.21.15.72:0/1238357292 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:01:58.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbebc877700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb80635d0 0x7fbeb80bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:58.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbebd078700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb81605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:58.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb7fff700 1 -- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 msgr2=0x7fbeb81605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:58.017 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb7fff700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb81605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb7fff700 1 -- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb80635d0 msgr2=0x7fbeb80bbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:58.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb7fff700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb80635d0 0x7fbeb80bbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb7fff700 1 -- 172.21.15.72:0/1238357292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea4009370 con 0x7fbeb80658a0 2023-10-18T18:01:58.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb7fff700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbeb80658a0 0x7fbeb81600d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fbea4006fd0 tx=0x7fbea4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:58.018 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb4ff9700 1 -- 172.21.15.72:0/1238357292 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400be90 con 0x7fbeb80658a0 2023-10-18T18:01:58.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb8163640 con 0x7fbeb80658a0 2023-10-18T18:01:58.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb4ff9700 1 -- 172.21.15.72:0/1238357292 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea400b7f0 con 0x7fbeb80658a0 2023-10-18T18:01:58.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb4ff9700 1 -- 172.21.15.72:0/1238357292 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea401c630 con 0x7fbeb80658a0 2023-10-18T18:01:58.019 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb8163c00 con 0x7fbeb80658a0 2023-10-18T18:01:58.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb8046f00 con 0x7fbeb80658a0 2023-10-18T18:01:58.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.009+0000 7fbeb4ff9700 1 -- 172.21.15.72:0/1238357292 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fbea401ca60 con 0x7fbeb80658a0 2023-10-18T18:01:58.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.013+0000 7fbeb4ff9700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbe9c05a4e0 0x7fbe9c05c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:01:58.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.013+0000 7fbeb4ff9700 1 -- 172.21.15.72:0/1238357292 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fbea4076620 con 0x7fbeb80658a0 2023-10-18T18:01:58.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.013+0000 7fbebc877700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbe9c05a4e0 0x7fbe9c05c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:01:58.021 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.013+0000 7fbebc877700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbe9c05a4e0 0x7fbe9c05c9a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fbeac009c30 tx=0x7fbeac009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:01:58.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.021+0000 7fbeb4ff9700 1 -- 172.21.15.72:0/1238357292 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fbea4049b70 con 0x7fbeb80658a0 2023-10-18T18:01:58.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbeb805e770 con 0x7fbeb80658a0 2023-10-18T18:01:58.218 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbeb4ff9700 1 -- 172.21.15.72:0/1238357292 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fbea4029080 con 0x7fbeb80658a0 2023-10-18T18:01:58.218 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:01:58.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbe9c05a4e0 msgr2=0x7fbe9c05c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:58.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbe9c05a4e0 0x7fbe9c05c9a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fbeac009c30 tx=0x7fbeac009060).stop 2023-10-18T18:01:58.220 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbeb80658a0 msgr2=0x7fbeb81600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:01:58.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbeb80658a0 0x7fbeb81600d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fbea4006fd0 tx=0x7fbea4017000).stop 2023-10-18T18:01:58.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 shutdown_connections 2023-10-18T18:01:58.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fbe9c05a4e0 0x7fbe9c05c9a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fbeb80c2bf0 0x7fbeb81605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.221 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbeb80658a0 0x7fbeb81600d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 --2- 172.21.15.72:0/1238357292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbeb80635d0 0x7fbeb80bbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:01:58.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.213+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 >> 172.21.15.72:0/1238357292 conn(0x7fbeb8062ba0 msgr2=0x7fbeb80c0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:01:58.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.217+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 shutdown_connections 2023-10-18T18:01:58.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:01:58.217+0000 7fbebd879700 1 -- 172.21.15.72:0/1238357292 wait complete. 2023-10-18T18:01:58.240 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:01:58.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~207s 2023-10-18T18:01:58.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~153s 2023-10-18T18:01:58.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~107s 2023-10-18T18:01:58.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~133s 2023-10-18T18:01:58.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~120s 2023-10-18T18:01:58.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~166s 2023-10-18T18:02:01.241 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:01.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.401+0000 7efc89379700 1 Processor -- start 2023-10-18T18:02:01.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.401+0000 7efc89379700 1 -- start start 2023-10-18T18:02:01.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc89379700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc8406a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:01.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc89379700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7efc84067d30 0x7efc840630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:01.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc89379700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efc84068150 0x7efc840635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:01.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc89379700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc84150c10 con 0x7efc84068150 2023-10-18T18:02:01.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc89379700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc84068910 con 0x7efc84067d30 2023-10-18T18:02:01.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc89379700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc84150780 con 0x7efc8414d170 2023-10-18T18:02:01.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc88b78700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efc84068150 0x7efc840635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:01.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc88b78700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efc84068150 0x7efc840635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47534/0 (socket says 172.21.15.72:47534) 2023-10-18T18:02:01.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc88b78700 1 -- 172.21.15.72:0/2115588113 learned_addr learned my addr 172.21.15.72:0/2115588113 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:01.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc7b7fe700 1 --2- 172.21.15.72:0/2115588113 >> v2:172.21.15.179:3300/0 conn(0x7efc84067d30 0x7efc840630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:01.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc83fff700 1 --2- 172.21.15.72:0/2115588113 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc8406a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:01.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc88b78700 1 -- 172.21.15.72:0/2115588113 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 msgr2=0x7efc8406a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc88b78700 1 --2- 172.21.15.72:0/2115588113 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc8406a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc88b78700 1 -- 172.21.15.72:0/2115588113 >> v2:172.21.15.179:3300/0 conn(0x7efc84067d30 msgr2=0x7efc840630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc88b78700 1 --2- 172.21.15.72:0/2115588113 >> v2:172.21.15.179:3300/0 conn(0x7efc84067d30 0x7efc840630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc88b78700 1 -- 172.21.15.72:0/2115588113 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc84068630 con 0x7efc84068150 2023-10-18T18:02:01.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc7b7fe700 1 --2- 172.21.15.72:0/2115588113 >> v2:172.21.15.179:3300/0 conn(0x7efc84067d30 0x7efc840630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:01.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc88b78700 1 --2- 172.21.15.72:0/2115588113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 0x7efc840635d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7efc84067460 tx=0x7efc7c009840).ready entity=mon.0 client_cookie=c589a9b18ba43cec server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:01.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc837fe700 1 -- 172.21.15.72:0/2115588113 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00e070 con 0x7efc84068150 2023-10-18T18:02:01.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc837fe700 1 -- 172.21.15.72:0/2115588113 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c00ba00 con 0x7efc84068150 2023-10-18T18:02:01.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.405+0000 7efc837fe700 1 -- 172.21.15.72:0/2115588113 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00bd80 con 0x7efc84068150 2023-10-18T18:02:01.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.409+0000 7efc89379700 1 -- 172.21.15.72:0/2115588113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 msgr2=0x7efc840635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:01.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.409+0000 7efc89379700 1 --2- 172.21.15.72:0/2115588113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 0x7efc840635d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7efc84067460 tx=0x7efc7c009840).stop 2023-10-18T18:02:01.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.409+0000 7efc89379700 1 -- 172.21.15.72:0/2115588113 shutdown_connections 2023-10-18T18:02:01.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.409+0000 7efc89379700 1 --2- 172.21.15.72:0/2115588113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 0x7efc840635d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.409+0000 7efc89379700 1 --2- 172.21.15.72:0/2115588113 >> v2:172.21.15.179:3300/0 conn(0x7efc84067d30 0x7efc840630e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.409+0000 7efc89379700 1 --2- 172.21.15.72:0/2115588113 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc8406a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.409+0000 7efc89379700 1 -- 172.21.15.72:0/2115588113 >> 172.21.15.72:0/2115588113 conn(0x7efc84062ba0 msgr2=0x7efc840c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:01.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.409+0000 7efc89379700 1 -- 172.21.15.72:0/2115588113 shutdown_connections 2023-10-18T18:02:01.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.409+0000 7efc89379700 1 -- 172.21.15.72:0/2115588113 wait complete. 2023-10-18T18:02:01.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc89379700 1 Processor -- start 2023-10-18T18:02:01.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc89379700 1 -- start start 2023-10-18T18:02:01.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc89379700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc84067d30 0x7efc841510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:01.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc89379700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 0x7efc8414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:01.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc89379700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc841603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:01.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc89379700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8415ff80 con 0x7efc84068150 2023-10-18T18:02:01.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc89379700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8415fc70 con 0x7efc84067d30 2023-10-18T18:02:01.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc89379700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c60e0 con 0x7efc8414d170 2023-10-18T18:02:01.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc7b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 0x7efc8414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:01.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc88b78700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc841603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:01.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc7b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 0x7efc8414d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47550/0 (socket says 172.21.15.72:47550) 2023-10-18T18:02:01.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc7b7fe700 1 -- 172.21.15.72:0/952814493 learned_addr learned my addr 172.21.15.72:0/952814493 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:01.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc88b78700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc841603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:46070/0 (socket says 172.21.15.72:46070) 2023-10-18T18:02:01.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc83fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc84067d30 0x7efc841510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:01.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc88b78700 1 -- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc84067d30 msgr2=0x7efc841510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:01.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc88b78700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc84067d30 0x7efc841510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc88b78700 1 -- 172.21.15.72:0/952814493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 msgr2=0x7efc8414d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc88b78700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 0x7efc8414d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc88b78700 1 -- 172.21.15.72:0/952814493 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c0092a0 con 0x7efc8414d170 2023-10-18T18:02:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc7b7fe700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 0x7efc8414d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.413+0000 7efc88b78700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc841603a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7efc7c013000 tx=0x7efc7c009840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:01.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.417+0000 7efc80ff9700 1 -- 172.21.15.72:0/952814493 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c013e60 con 0x7efc8414d170 2023-10-18T18:02:01.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.417+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7c009ae0 con 0x7efc8414d170 2023-10-18T18:02:01.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.417+0000 7efc80ff9700 1 -- 172.21.15.72:0/952814493 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c0092e0 con 0x7efc8414d170 2023-10-18T18:02:01.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.417+0000 7efc80ff9700 1 -- 172.21.15.72:0/952814493 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c01e6c0 con 0x7efc8414d170 2023-10-18T18:02:01.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.417+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc84151b40 con 0x7efc8414d170 2023-10-18T18:02:01.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.417+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc841600c0 con 0x7efc8414d170 2023-10-18T18:02:01.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.417+0000 7efc80ff9700 1 -- 172.21.15.72:0/952814493 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efc7c003040 con 0x7efc8414d170 2023-10-18T18:02:01.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.421+0000 7efc80ff9700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efc6405a540 0x7efc6405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.421+0000 7efc80ff9700 1 -- 172.21.15.72:0/952814493 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7efc7c075d70 con 0x7efc8414d170 2023-10-18T18:02:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.421+0000 7efc83fff700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efc6405a540 0x7efc6405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:01.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.421+0000 7efc83fff700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efc6405a540 0x7efc6405ca00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7efc74009b90 tx=0x7efc74009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:01.435 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.429+0000 7efc80ff9700 1 -- 172.21.15.72:0/952814493 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7efc7c049370 con 0x7efc8414d170 2023-10-18T18:02:01.631 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.625+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efc84046f00 con 0x7efc8414d170 2023-10-18T18:02:01.632 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.625+0000 7efc80ff9700 1 -- 172.21.15.72:0/952814493 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7efc7c02c070 con 0x7efc8414d170 2023-10-18T18:02:01.633 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:01.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efc6405a540 msgr2=0x7efc6405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:01.636 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efc6405a540 0x7efc6405ca00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7efc74009b90 tx=0x7efc74009060).stop 2023-10-18T18:02:01.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 msgr2=0x7efc841603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:01.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc841603a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7efc7c013000 tx=0x7efc7c009840).stop 2023-10-18T18:02:01.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 shutdown_connections 2023-10-18T18:02:01.637 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efc6405a540 0x7efc6405ca00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efc8414d170 0x7efc841603a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efc84068150 0x7efc8414d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 --2- 172.21.15.72:0/952814493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc84067d30 0x7efc841510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:01.638 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.629+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 >> 172.21.15.72:0/952814493 conn(0x7efc84062ba0 msgr2=0x7efc84063240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:01.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.633+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 shutdown_connections 2023-10-18T18:02:01.639 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:01.633+0000 7efc89379700 1 -- 172.21.15.72:0/952814493 wait complete. 2023-10-18T18:02:01.655 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:04.657 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:04.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~214s 2023-10-18T18:02:04.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~159s 2023-10-18T18:02:04.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~114s 2023-10-18T18:02:04.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~139s 2023-10-18T18:02:04.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~126s 2023-10-18T18:02:04.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~173s 2023-10-18T18:02:04.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.821+0000 7f9a4f0db700 1 Processor -- start 2023-10-18T18:02:04.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.825+0000 7f9a4f0db700 1 -- start start 2023-10-18T18:02:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.825+0000 7f9a4f0db700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9a480bca50 0x7f9a480b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.825+0000 7f9a4f0db700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9a48065d30 0x7f9a4814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.825+0000 7f9a4f0db700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a480bc480 con 0x7f9a480b9690 2023-10-18T18:02:04.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.825+0000 7f9a4f0db700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a48065190 con 0x7f9a480bca50 2023-10-18T18:02:04.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.825+0000 7f9a4f0db700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a480bb920 con 0x7f9a48065d30 2023-10-18T18:02:04.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.825+0000 7f9a4ce77700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f9a480b9690 0x7f9a48065620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36532/0 (socket says 172.21.15.72:36532) 2023-10-18T18:02:04.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.825+0000 7f9a4ce77700 1 -- 172.21.15.72:0/1219524012 learned_addr learned my addr 172.21.15.72:0/1219524012 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:04.834 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4d678700 1 --2- 172.21.15.72:0/1219524012 >> v2:172.21.15.179:3300/0 conn(0x7f9a48065d30 0x7f9a4814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:04.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a47fff700 1 --2- 172.21.15.72:0/1219524012 >> v2:172.21.15.72:3300/0 conn(0x7f9a480bca50 0x7f9a480b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:04.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a477fe700 1 -- 172.21.15.72:0/1219524012 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136505567 0 0) 0x7f9a480bc480 con 0x7f9a480b9690 2023-10-18T18:02:04.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a477fe700 1 -- 172.21.15.72:0/1219524012 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a380033f0 con 0x7f9a480b9690 2023-10-18T18:02:04.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a47fff700 1 -- 172.21.15.72:0/1219524012 >> v2:172.21.15.179:3300/0 conn(0x7f9a48065d30 msgr2=0x7f9a4814dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:04.835 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a47fff700 1 --2- 172.21.15.72:0/1219524012 >> v2:172.21.15.179:3300/0 conn(0x7f9a48065d30 0x7f9a4814dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:04.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a47fff700 1 -- 172.21.15.72:0/1219524012 >> v1:172.21.15.72:6789/0 conn(0x7f9a480b9690 legacy=0x7f9a48065620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:04.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a47fff700 1 -- 172.21.15.72:0/1219524012 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a480bbbe0 con 0x7f9a480bca50 2023-10-18T18:02:04.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4d678700 1 --2- 172.21.15.72:0/1219524012 >> v2:172.21.15.179:3300/0 conn(0x7f9a48065d30 0x7f9a4814dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:04.836 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a47fff700 1 --2- 172.21.15.72:0/1219524012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a480bca50 0x7f9a480b9a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9a480470c0 tx=0x7f9a34009060).ready entity=mon.0 client_cookie=65084bf573464558 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a477fe700 1 -- 172.21.15.72:0/1219524012 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400d4a0 con 0x7f9a480bca50 2023-10-18T18:02:04.837 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a477fe700 1 -- 172.21.15.72:0/1219524012 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3400d600 con 0x7f9a480bca50 2023-10-18T18:02:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a477fe700 1 -- 172.21.15.72:0/1219524012 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400d980 con 0x7f9a480bca50 2023-10-18T18:02:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4f0db700 1 -- 172.21.15.72:0/1219524012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a480bca50 msgr2=0x7f9a480b9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:04.838 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4f0db700 1 --2- 172.21.15.72:0/1219524012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a480bca50 0x7f9a480b9a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9a480470c0 tx=0x7f9a34009060).stop 2023-10-18T18:02:04.842 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4f0db700 1 -- 172.21.15.72:0/1219524012 shutdown_connections 2023-10-18T18:02:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4f0db700 1 --2- 172.21.15.72:0/1219524012 >> v2:172.21.15.179:3300/0 conn(0x7f9a48065d30 0x7f9a4814dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:04.843 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4f0db700 1 --2- 172.21.15.72:0/1219524012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a480bca50 0x7f9a480b9a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4f0db700 1 -- 172.21.15.72:0/1219524012 >> 172.21.15.72:0/1219524012 conn(0x7f9a480620d0 msgr2=0x7f9a480624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4f0db700 1 -- 172.21.15.72:0/1219524012 shutdown_connections 2023-10-18T18:02:04.844 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.829+0000 7f9a4f0db700 1 -- 172.21.15.72:0/1219524012 wait complete. 2023-10-18T18:02:04.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.833+0000 7f9a4f0db700 1 Processor -- start 2023-10-18T18:02:04.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.833+0000 7f9a4f0db700 1 -- start start 2023-10-18T18:02:04.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4f0db700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a48065d30 0x7f9a480c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:04.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4f0db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a480b9690 0x7f9a48167da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:04.846 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4f0db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a480c2bf0 0x7f9a48168290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4f0db700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a48069ea0 con 0x7f9a48065d30 2023-10-18T18:02:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4f0db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a48168780 con 0x7f9a480c2bf0 2023-10-18T18:02:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4f0db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4814c960 con 0x7f9a480b9690 2023-10-18T18:02:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4ce77700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a48065d30 0x7f9a480c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:04.847 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4ce77700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a48065d30 0x7f9a480c2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47578/0 (socket says 172.21.15.72:47578) 2023-10-18T18:02:04.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a47fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a480b9690 0x7f9a48167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:04.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4ce77700 1 -- 172.21.15.72:0/405810801 learned_addr learned my addr 172.21.15.72:0/405810801 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:04.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4d678700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a480c2bf0 0x7f9a48168290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:04.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4ce77700 1 -- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a480b9690 msgr2=0x7f9a48167da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:04.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4ce77700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a480b9690 0x7f9a48167da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:04.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4ce77700 1 -- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a480c2bf0 msgr2=0x7f9a48168290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:04.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4ce77700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a480c2bf0 0x7f9a48168290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:04.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4ce77700 1 -- 172.21.15.72:0/405810801 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a40009a20 con 0x7f9a48065d30 2023-10-18T18:02:04.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4ce77700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a48065d30 0x7f9a480c2700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f9a3c00c590 tx=0x7f9a3c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:04.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a44ff9700 1 -- 172.21.15.72:0/405810801 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c007e40 con 0x7f9a48065d30 2023-10-18T18:02:04.849 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a44ff9700 1 -- 172.21.15.72:0/405810801 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c0072e0 con 0x7f9a48065d30 2023-10-18T18:02:04.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a44ff9700 1 -- 172.21.15.72:0/405810801 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0057a0 con 0x7f9a48065d30 2023-10-18T18:02:04.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a340092a0 con 0x7f9a48065d30 2023-10-18T18:02:04.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.837+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a48069570 con 0x7f9a48065d30 2023-10-18T18:02:04.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.841+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4806c190 con 0x7f9a48065d30 2023-10-18T18:02:04.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.841+0000 7f9a44ff9700 1 -- 172.21.15.72:0/405810801 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f9a3c005bd0 con 0x7f9a48065d30 2023-10-18T18:02:04.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.841+0000 7f9a44ff9700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9a2c05a510 0x7f9a2c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.841+0000 7f9a44ff9700 1 -- 172.21.15.72:0/405810801 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f9a3c0742e0 con 0x7f9a48065d30 2023-10-18T18:02:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.841+0000 7f9a47fff700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9a2c05a510 0x7f9a2c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:04.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.845+0000 7f9a47fff700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9a2c05a510 0x7f9a2c05c9d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f9a480470c0 tx=0x7f9a34016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:04.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:04.849+0000 7f9a44ff9700 1 -- 172.21.15.72:0/405810801 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f9a3c047830 con 0x7f9a48065d30 2023-10-18T18:02:05.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.041+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9a480bbec0 con 0x7f9a48065d30 2023-10-18T18:02:05.049 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.041+0000 7f9a44ff9700 1 -- 172.21.15.72:0/405810801 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f9a3c04ae50 con 0x7f9a48065d30 2023-10-18T18:02:05.050 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:05.051 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9a2c05a510 msgr2=0x7f9a2c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:05.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9a2c05a510 0x7f9a2c05c9d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f9a480470c0 tx=0x7f9a34016000).stop 2023-10-18T18:02:05.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a48065d30 msgr2=0x7f9a480c2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:05.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a48065d30 0x7f9a480c2700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f9a3c00c590 tx=0x7f9a3c007c80).stop 2023-10-18T18:02:05.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 shutdown_connections 2023-10-18T18:02:05.052 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f9a2c05a510 0x7f9a2c05c9d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:05.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a480c2bf0 0x7f9a48168290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:05.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9a480b9690 0x7f9a48167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:05.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 --2- 172.21.15.72:0/405810801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9a48065d30 0x7f9a480c2700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:05.053 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 >> 172.21.15.72:0/405810801 conn(0x7f9a480620d0 msgr2=0x7f9a48153e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:05.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 shutdown_connections 2023-10-18T18:02:05.054 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:05.045+0000 7f9a4f0db700 1 -- 172.21.15.72:0/405810801 wait complete. 2023-10-18T18:02:05.073 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:08.074 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:08.235 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.229+0000 7f0364a39700 1 Processor -- start 2023-10-18T18:02:08.236 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.229+0000 7f0364a39700 1 -- start start 2023-10-18T18:02:08.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f0364a39700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f03601498d0 0x7f03600bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:08.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f0364a39700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f03600bc930 0x7f03600bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:08.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f0364a39700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600be3a0 0x7f0360063350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:08.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f0364a39700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03600be810 con 0x7f03600bc930 2023-10-18T18:02:08.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f0364a39700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036014c380 con 0x7f03601498d0 2023-10-18T18:02:08.240 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f0364a39700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036014c640 con 0x7f03600be3a0 2023-10-18T18:02:08.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f035ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600be3a0 0x7f0360063350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:08.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f035ffff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600be3a0 0x7f0360063350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:46076/0 (socket says 172.21.15.72:46076) 2023-10-18T18:02:08.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f035ffff700 1 -- 172.21.15.72:0/4222789855 learned_addr learned my addr 172.21.15.72:0/4222789855 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:08.241 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f035f7fe700 1 --2- 172.21.15.72:0/4222789855 >> v2:172.21.15.179:3300/0 conn(0x7f03601498d0 0x7f03600bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:08.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f0356ffd700 1 --2- 172.21.15.72:0/4222789855 >> v2:172.21.15.72:3300/0 conn(0x7f03600bc930 0x7f03600bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:08.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f035ffff700 1 -- 172.21.15.72:0/4222789855 >> v2:172.21.15.179:3300/0 conn(0x7f03601498d0 msgr2=0x7f03600bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:08.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f035ffff700 1 --2- 172.21.15.72:0/4222789855 >> v2:172.21.15.179:3300/0 conn(0x7f03601498d0 0x7f03600bd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f035ffff700 1 -- 172.21.15.72:0/4222789855 >> v2:172.21.15.72:3300/0 conn(0x7f03600bc930 msgr2=0x7f03600bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:08.242 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f035ffff700 1 --2- 172.21.15.72:0/4222789855 >> v2:172.21.15.72:3300/0 conn(0x7f03600bc930 0x7f03600bdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.233+0000 7f035ffff700 1 -- 172.21.15.72:0/4222789855 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036014c920 con 0x7f03600be3a0 2023-10-18T18:02:08.243 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f035ffff700 1 --2- 172.21.15.72:0/4222789855 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600be3a0 0x7f0360063350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f036006bbf0 tx=0x7f0358009840).ready entity=mon.2 client_cookie=b99fecf0fe7c9c42 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:08.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f035effd700 1 -- 172.21.15.72:0/4222789855 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035800e070 con 0x7f03600be3a0 2023-10-18T18:02:08.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f035effd700 1 -- 172.21.15.72:0/4222789855 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035800ba20 con 0x7f03600be3a0 2023-10-18T18:02:08.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f035effd700 1 -- 172.21.15.72:0/4222789855 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035800bda0 con 0x7f03600be3a0 2023-10-18T18:02:08.244 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f0364a39700 1 -- 172.21.15.72:0/4222789855 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600be3a0 msgr2=0x7f0360063350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:08.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f0364a39700 1 --2- 172.21.15.72:0/4222789855 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600be3a0 0x7f0360063350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f036006bbf0 tx=0x7f0358009840).stop 2023-10-18T18:02:08.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f0364a39700 1 -- 172.21.15.72:0/4222789855 shutdown_connections 2023-10-18T18:02:08.245 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f0364a39700 1 --2- 172.21.15.72:0/4222789855 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600be3a0 0x7f0360063350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f0364a39700 1 --2- 172.21.15.72:0/4222789855 >> v2:172.21.15.72:3300/0 conn(0x7f03600bc930 0x7f03600bdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f0364a39700 1 --2- 172.21.15.72:0/4222789855 >> v2:172.21.15.179:3300/0 conn(0x7f03601498d0 0x7f03600bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f0364a39700 1 -- 172.21.15.72:0/4222789855 >> 172.21.15.72:0/4222789855 conn(0x7f0360062ba0 msgr2=0x7f03600c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:08.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f0364a39700 1 -- 172.21.15.72:0/4222789855 shutdown_connections 2023-10-18T18:02:08.246 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.237+0000 7f0364a39700 1 -- 172.21.15.72:0/4222789855 wait complete. 2023-10-18T18:02:08.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0364a39700 1 Processor -- start 2023-10-18T18:02:08.247 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0364a39700 1 -- start start 2023-10-18T18:02:08.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0364a39700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600bc930 0x7f0360161860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:08.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0364a39700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03600be3a0 0x7f0360063940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:08.248 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0364a39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03601498d0 0x7f03600b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0364a39700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03600b7180 con 0x7f03600be3a0 2023-10-18T18:02:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0364a39700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036015dac0 con 0x7f03601498d0 2023-10-18T18:02:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0364a39700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036015dda0 con 0x7f03600bc930 2023-10-18T18:02:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0356ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03600be3a0 0x7f0360063940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:08.249 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0356ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03600be3a0 0x7f0360063940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:47598/0 (socket says 172.21.15.72:47598) 2023-10-18T18:02:08.250 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.241+0000 7f0356ffd700 1 -- 172.21.15.72:0/1517290810 learned_addr learned my addr 172.21.15.72:0/1517290810 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:08.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f035f7fe700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600bc930 0x7f0360161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:08.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f035ffff700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03601498d0 0x7f03600b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:08.251 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0356ffd700 1 -- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600bc930 msgr2=0x7f0360161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0356ffd700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600bc930 0x7f0360161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0356ffd700 1 -- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03601498d0 msgr2=0x7f03600b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0356ffd700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03601498d0 0x7f03600b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0356ffd700 1 -- 172.21.15.72:0/1517290810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358009ae0 con 0x7f03600be3a0 2023-10-18T18:02:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0356ffd700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03600be3a0 0x7f0360063940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f034800d890 tx=0x7f0348008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0357fff700 1 -- 172.21.15.72:0/1517290810 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348011590 con 0x7f03600be3a0 2023-10-18T18:02:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0357fff700 1 -- 172.21.15.72:0/1517290810 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03480116f0 con 0x7f03600be3a0 2023-10-18T18:02:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036015e080 con 0x7f03600be3a0 2023-10-18T18:02:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0357fff700 1 -- 172.21.15.72:0/1517290810 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348011a70 con 0x7f03600be3a0 2023-10-18T18:02:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036015e640 con 0x7f03600be3a0 2023-10-18T18:02:08.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.245+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0360046f00 con 0x7f03600be3a0 2023-10-18T18:02:08.254 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.249+0000 7f0357fff700 1 -- 172.21.15.72:0/1517290810 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f034801c460 con 0x7f03600be3a0 2023-10-18T18:02:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.249+0000 7f0357fff700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f034005a510 0x7f034005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:08.255 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.249+0000 7f0357fff700 1 -- 172.21.15.72:0/1517290810 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f03480746e0 con 0x7f03600be3a0 2023-10-18T18:02:08.256 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.249+0000 7f035f7fe700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f034005a510 0x7f034005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:08.257 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.253+0000 7f035f7fe700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f034005a510 0x7f034005c9d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0350009c30 tx=0x7f0350009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:08.262 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.257+0000 7f0357fff700 1 -- 172.21.15.72:0/1517290810 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f0348047bd0 con 0x7f03600be3a0 2023-10-18T18:02:08.460 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.453+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f03600bd150 con 0x7f03600be3a0 2023-10-18T18:02:08.461 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.457+0000 7f0357fff700 1 -- 172.21.15.72:0/1517290810 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f0348011e60 con 0x7f03600be3a0 2023-10-18T18:02:08.461 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:08.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f034005a510 msgr2=0x7f034005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:08.466 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f034005a510 0x7f034005c9d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0350009c30 tx=0x7f0350009060).stop 2023-10-18T18:02:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03600be3a0 msgr2=0x7f0360063940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03600be3a0 0x7f0360063940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f034800d890 tx=0x7f0348008040).stop 2023-10-18T18:02:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 shutdown_connections 2023-10-18T18:02:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f034005a510 0x7f034005c9d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.467 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f03601498d0 0x7f03600b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03600be3a0 0x7f0360063940 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 --2- 172.21.15.72:0/1517290810 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f03600bc930 0x7f0360161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:08.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 >> 172.21.15.72:0/1517290810 conn(0x7f0360062ba0 msgr2=0x7f0360068a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:08.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 shutdown_connections 2023-10-18T18:02:08.468 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:08.461+0000 7f0364a39700 1 -- 172.21.15.72:0/1517290810 wait complete. 2023-10-18T18:02:08.486 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:11.046 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~220s 2023-10-18T18:02:11.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~166s 2023-10-18T18:02:11.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~120s 2023-10-18T18:02:11.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~146s 2023-10-18T18:02:11.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~133s 2023-10-18T18:02:11.047 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~179s 2023-10-18T18:02:11.488 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:11.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.645+0000 7fb235d07700 1 Processor -- start 2023-10-18T18:02:11.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.645+0000 7fb235d07700 1 -- start start 2023-10-18T18:02:11.654 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235d07700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb230065490 0x7fb2301510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:11.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235d07700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb2300b9130 0x7fb2301515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:11.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235d07700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23014b440 con 0x7fb23014b060 2023-10-18T18:02:11.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235d07700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2300bde20 con 0x7fb2300b9130 2023-10-18T18:02:11.655 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235d07700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c37a0 con 0x7fb230065490 2023-10-18T18:02:11.656 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235506700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb2300b9130 0x7fb2301515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:11.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb227fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb230065490 0x7fb2301510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:11.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235506700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fb2300b9130 0x7fb2301515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:35236/0 (socket says 172.21.15.72:35236) 2023-10-18T18:02:11.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb227fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb230065490 0x7fb2301510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:37744/0 (socket says 172.21.15.72:37744) 2023-10-18T18:02:11.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235506700 1 -- 172.21.15.72:0/3507426208 learned_addr learned my addr 172.21.15.72:0/3507426208 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:11.657 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235506700 1 -- 172.21.15.72:0/3507426208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb230065490 msgr2=0x7fb2301510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:11.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235506700 1 --2- 172.21.15.72:0/3507426208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb230065490 0x7fb2301510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:11.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235506700 1 -- 172.21.15.72:0/3507426208 >> v1:172.21.15.72:6789/0 conn(0x7fb23014b060 legacy=0x7fb230064d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:11.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235506700 1 -- 172.21.15.72:0/3507426208 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230151ab0 con 0x7fb2300b9130 2023-10-18T18:02:11.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb227fff700 1 --2- 172.21.15.72:0/3507426208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb230065490 0x7fb2301510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:11.658 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.649+0000 7fb235506700 1 --2- 172.21.15.72:0/3507426208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 0x7fb2301515c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb228009f20 tx=0x7fb228009840).ready entity=mon.1 client_cookie=508da08042137fdf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:11.659 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb2277fe700 1 -- 172.21.15.72:0/3507426208 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800e070 con 0x7fb2300b9130 2023-10-18T18:02:11.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb2277fe700 1 -- 172.21.15.72:0/3507426208 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22800b9d0 con 0x7fb2300b9130 2023-10-18T18:02:11.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb2277fe700 1 -- 172.21.15.72:0/3507426208 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800bd50 con 0x7fb2300b9130 2023-10-18T18:02:11.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb235d07700 1 -- 172.21.15.72:0/3507426208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 msgr2=0x7fb2301515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:11.660 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb235d07700 1 --2- 172.21.15.72:0/3507426208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 0x7fb2301515c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fb228009f20 tx=0x7fb228009840).stop 2023-10-18T18:02:11.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb235d07700 1 -- 172.21.15.72:0/3507426208 shutdown_connections 2023-10-18T18:02:11.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb235d07700 1 --2- 172.21.15.72:0/3507426208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 0x7fb2301515c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:11.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb235d07700 1 --2- 172.21.15.72:0/3507426208 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb230065490 0x7fb2301510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:11.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb235d07700 1 -- 172.21.15.72:0/3507426208 >> 172.21.15.72:0/3507426208 conn(0x7fb230069d80 msgr2=0x7fb23006a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:11.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb235d07700 1 -- 172.21.15.72:0/3507426208 shutdown_connections 2023-10-18T18:02:11.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.653+0000 7fb235d07700 1 -- 172.21.15.72:0/3507426208 wait complete. 2023-10-18T18:02:11.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb235d07700 1 Processor -- start 2023-10-18T18:02:11.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb235d07700 1 -- start start 2023-10-18T18:02:11.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb235d07700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb230065490 0x7fb230161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb235d07700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 0x7fb230153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb235d07700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb23014b060 0x7fb23015daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb235d07700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c08b0 con 0x7fb230065490 2023-10-18T18:02:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb235d07700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23015df90 con 0x7fb2300b9130 2023-10-18T18:02:11.664 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb235d07700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb230161110 con 0x7fb23014b060 2023-10-18T18:02:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb234d05700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb230065490 0x7fb230161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb234d05700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb230065490 0x7fb230161bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45940/0 (socket says 172.21.15.72:45940) 2023-10-18T18:02:11.665 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb234d05700 1 -- 172.21.15.72:0/3660015798 learned_addr learned my addr 172.21.15.72:0/3660015798 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:11.666 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb227fff700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 0x7fb230153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb235506700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb23014b060 0x7fb23015daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb234d05700 1 -- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb23014b060 msgr2=0x7fb23015daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb234d05700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb23014b060 0x7fb23015daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:11.667 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb234d05700 1 -- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 msgr2=0x7fb230153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb234d05700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 0x7fb230153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb234d05700 1 -- 172.21.15.72:0/3660015798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2200092a0 con 0x7fb230065490 2023-10-18T18:02:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.657+0000 7fb227fff700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 0x7fb230153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:02:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.661+0000 7fb234d05700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb230065490 0x7fb230161bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fb22c00c4c0 tx=0x7fb22c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.661+0000 7fb224ff9700 1 -- 172.21.15.72:0/3660015798 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c007020 con 0x7fb230065490 2023-10-18T18:02:11.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.661+0000 7fb224ff9700 1 -- 172.21.15.72:0/3660015798 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c001e90 con 0x7fb230065490 2023-10-18T18:02:11.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.661+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb228009ae0 con 0x7fb230065490 2023-10-18T18:02:11.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.661+0000 7fb224ff9700 1 -- 172.21.15.72:0/3660015798 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0073d0 con 0x7fb230065490 2023-10-18T18:02:11.669 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.661+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb230154840 con 0x7fb230065490 2023-10-18T18:02:11.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.661+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb230046f00 con 0x7fb230065490 2023-10-18T18:02:11.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.661+0000 7fb224ff9700 1 -- 172.21.15.72:0/3660015798 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fb22c0068b0 con 0x7fb230065490 2023-10-18T18:02:11.670 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.665+0000 7fb224ff9700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb21c05a510 0x7fb21c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:11.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.665+0000 7fb224ff9700 1 -- 172.21.15.72:0/3660015798 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fb22c0740b0 con 0x7fb230065490 2023-10-18T18:02:11.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.665+0000 7fb227fff700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb21c05a510 0x7fb21c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:11.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.665+0000 7fb227fff700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb21c05a510 0x7fb21c05c9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb220009270 tx=0x7fb220008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:11.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.669+0000 7fb224ff9700 1 -- 172.21.15.72:0/3660015798 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fb22c0476c0 con 0x7fb230065490 2023-10-18T18:02:11.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.861+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb23014b440 con 0x7fb230065490 2023-10-18T18:02:11.868 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.861+0000 7fb224ff9700 1 -- 172.21.15.72:0/3660015798 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fb23014b440 con 0x7fb230065490 2023-10-18T18:02:11.869 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:11.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb21c05a510 msgr2=0x7fb21c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:11.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb21c05a510 0x7fb21c05c9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fb220009270 tx=0x7fb220008040).stop 2023-10-18T18:02:11.871 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb230065490 msgr2=0x7fb230161bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:11.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb230065490 0x7fb230161bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fb22c00c4c0 tx=0x7fb22c007e00).stop 2023-10-18T18:02:11.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 shutdown_connections 2023-10-18T18:02:11.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fb21c05a510 0x7fb21c05c9d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:11.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fb23014b060 0x7fb23015daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:11.872 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2300b9130 0x7fb230153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:11.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 --2- 172.21.15.72:0/3660015798 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb230065490 0x7fb230161bb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:11.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 >> 172.21.15.72:0/3660015798 conn(0x7fb230069d80 msgr2=0x7fb2300bef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:11.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 shutdown_connections 2023-10-18T18:02:11.874 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:11.865+0000 7fb235d07700 1 -- 172.21.15.72:0/3660015798 wait complete. 2023-10-18T18:02:11.894 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:14.895 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:15.065 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4b0161b700 1 Processor -- start 2023-10-18T18:02:15.066 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4b0161b700 1 -- start start 2023-10-18T18:02:15.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4b0161b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc062f80 0x7f4afc15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:15.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4b0161b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc063470 con 0x7f4afc14d450 2023-10-18T18:02:15.067 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4b0161b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc0c4410 con 0x7f4afc0c2be0 2023-10-18T18:02:15.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4b0161b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0c46f0 con 0x7f4afc062f80 2023-10-18T18:02:15.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4afaffd700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f4afc14d450 0x7f4afc0c24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42460/0 (socket says 172.21.15.72:42460) 2023-10-18T18:02:15.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4afaffd700 1 -- 172.21.15.72:0/3604501329 learned_addr learned my addr 172.21.15.72:0/3604501329 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:15.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4afbfff700 1 --2- 172.21.15.72:0/3604501329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc062f80 0x7f4afc15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:15.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4af9ffb700 1 -- 172.21.15.72:0/3604501329 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849634381 0 0) 0x7f4afc0c4410 con 0x7f4afc0c2be0 2023-10-18T18:02:15.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4af9ffb700 1 -- 172.21.15.72:0/3604501329 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae40033f0 con 0x7f4afc0c2be0 2023-10-18T18:02:15.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4af9ffb700 1 -- 172.21.15.72:0/3604501329 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407466570 0 0) 0x7f4afc063470 con 0x7f4afc14d450 2023-10-18T18:02:15.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.061+0000 7f4af9ffb700 1 -- 172.21.15.72:0/3604501329 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4afc0c4410 con 0x7f4afc14d450 2023-10-18T18:02:15.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4afbfff700 1 -- 172.21.15.72:0/3604501329 >> v1:172.21.15.179:6789/0 conn(0x7f4afc0c2be0 legacy=0x7f4afc150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:15.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4afbfff700 1 -- 172.21.15.72:0/3604501329 >> v1:172.21.15.72:6789/0 conn(0x7f4afc14d450 legacy=0x7f4afc0c24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:15.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4afbfff700 1 -- 172.21.15.72:0/3604501329 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0c49d0 con 0x7f4afc062f80 2023-10-18T18:02:15.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4afbfff700 1 --2- 172.21.15.72:0/3604501329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc062f80 0x7f4afc15b8b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4af4009e90 tx=0x7f4af4009780).ready entity=mon.2 client_cookie=c96d9e7d51d03a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:15.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4af9ffb700 1 -- 172.21.15.72:0/3604501329 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af400e070 con 0x7f4afc062f80 2023-10-18T18:02:15.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4af9ffb700 1 -- 172.21.15.72:0/3604501329 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af400bab0 con 0x7f4afc062f80 2023-10-18T18:02:15.072 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4af9ffb700 1 -- 172.21.15.72:0/3604501329 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af400be30 con 0x7f4afc062f80 2023-10-18T18:02:15.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4b0161b700 1 -- 172.21.15.72:0/3604501329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc062f80 msgr2=0x7f4afc15b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:15.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4b0161b700 1 --2- 172.21.15.72:0/3604501329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc062f80 0x7f4afc15b8b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f4af4009e90 tx=0x7f4af4009780).stop 2023-10-18T18:02:15.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4b0161b700 1 -- 172.21.15.72:0/3604501329 shutdown_connections 2023-10-18T18:02:15.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4b0161b700 1 --2- 172.21.15.72:0/3604501329 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc062f80 0x7f4afc15b8b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:15.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4b0161b700 1 -- 172.21.15.72:0/3604501329 >> 172.21.15.72:0/3604501329 conn(0x7f4afc05dc20 msgr2=0x7f4afc05e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:15.073 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4b0161b700 1 -- 172.21.15.72:0/3604501329 shutdown_connections 2023-10-18T18:02:15.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.065+0000 7f4b0161b700 1 -- 172.21.15.72:0/3604501329 wait complete. 2023-10-18T18:02:15.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4b0161b700 1 Processor -- start 2023-10-18T18:02:15.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4b0161b700 1 -- start start 2023-10-18T18:02:15.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4b0161b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4afc062f80 0x7f4afc0bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:15.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4b0161b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc0c2be0 0x7f4afc160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:15.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4b0161b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc14d450 0x7f4afc0bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:15.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4b0161b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc1606b0 con 0x7f4afc062f80 2023-10-18T18:02:15.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4b0161b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc1513d0 con 0x7f4afc14d450 2023-10-18T18:02:15.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4b0161b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc151640 con 0x7f4afc0c2be0 2023-10-18T18:02:15.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4afc062f80 0x7f4afc0bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:15.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4afc062f80 0x7f4afc0bbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45956/0 (socket says 172.21.15.72:45956) 2023-10-18T18:02:15.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afaffd700 1 -- 172.21.15.72:0/3832516365 learned_addr learned my addr 172.21.15.72:0/3832516365 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:15.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afbfff700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc14d450 0x7f4afc0bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:15.078 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afa7fc700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc0c2be0 0x7f4afc160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:15.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afaffd700 1 -- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc0c2be0 msgr2=0x7f4afc160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:15.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afaffd700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc0c2be0 0x7f4afc160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:15.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afaffd700 1 -- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc14d450 msgr2=0x7f4afc0bfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:15.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afaffd700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc14d450 0x7f4afc0bfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:15.079 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afaffd700 1 -- 172.21.15.72:0/3832516365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af4009a20 con 0x7f4afc062f80 2023-10-18T18:02:15.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afa7fc700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc0c2be0 0x7f4afc160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:02:15.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4afaffd700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4afc062f80 0x7f4afc0bbc80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f4aec00c450 tx=0x7f4aec007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:15.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4ae2ffd700 1 -- 172.21.15.72:0/3832516365 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec00ce70 con 0x7f4afc062f80 2023-10-18T18:02:15.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4ae2ffd700 1 -- 172.21.15.72:0/3832516365 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec0072e0 con 0x7f4afc062f80 2023-10-18T18:02:15.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc151920 con 0x7f4afc062f80 2023-10-18T18:02:15.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.069+0000 7f4ae2ffd700 1 -- 172.21.15.72:0/3832516365 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0057a0 con 0x7f4afc062f80 2023-10-18T18:02:15.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.073+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc062a70 con 0x7f4afc062f80 2023-10-18T18:02:15.081 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.073+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc1604e0 con 0x7f4afc062f80 2023-10-18T18:02:15.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.073+0000 7f4ae2ffd700 1 -- 172.21.15.72:0/3832516365 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f4aec005bd0 con 0x7f4afc062f80 2023-10-18T18:02:15.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.073+0000 7f4ae2ffd700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f4ae805a510 0x7f4ae805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:15.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.073+0000 7f4ae2ffd700 1 -- 172.21.15.72:0/3832516365 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f4aec0740c0 con 0x7f4afc062f80 2023-10-18T18:02:15.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.077+0000 7f4afa7fc700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f4ae805a510 0x7f4ae805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:15.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.077+0000 7f4afa7fc700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f4ae805a510 0x7f4ae805c9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f4afc154500 tx=0x7f4af0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:15.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.081+0000 7f4ae2ffd700 1 -- 172.21.15.72:0/3832516365 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f4aec0476d0 con 0x7f4afc062f80 2023-10-18T18:02:15.285 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.281+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4afc06c190 con 0x7f4afc062f80 2023-10-18T18:02:15.286 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.281+0000 7f4ae2ffd700 1 -- 172.21.15.72:0/3832516365 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f4aec04acf0 con 0x7f4afc062f80 2023-10-18T18:02:15.286 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:15.290 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f4ae805a510 msgr2=0x7f4ae805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f4ae805a510 0x7f4ae805c9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f4afc154500 tx=0x7f4af0009040).stop 2023-10-18T18:02:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4afc062f80 msgr2=0x7f4afc0bbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4afc062f80 0x7f4afc0bbc80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f4aec00c450 tx=0x7f4aec007c80).stop 2023-10-18T18:02:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 shutdown_connections 2023-10-18T18:02:15.291 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f4ae805a510 0x7f4ae805c9d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4afc14d450 0x7f4afc0bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f4afc0c2be0 0x7f4afc160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 --2- 172.21.15.72:0/3832516365 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4afc062f80 0x7f4afc0bbc80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 >> 172.21.15.72:0/3832516365 conn(0x7f4afc05dc20 msgr2=0x7f4afc14e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:15.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 shutdown_connections 2023-10-18T18:02:15.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:15.285+0000 7f4b0161b700 1 -- 172.21.15.72:0/3832516365 wait complete. 2023-10-18T18:02:15.306 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:17.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~226s 2023-10-18T18:02:17.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~172s 2023-10-18T18:02:17.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~126s 2023-10-18T18:02:17.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~152s 2023-10-18T18:02:17.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~139s 2023-10-18T18:02:17.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~185s 2023-10-18T18:02:18.307 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:18.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf6679700 1 Processor -- start 2023-10-18T18:02:18.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf6679700 1 -- start start 2023-10-18T18:02:18.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf6679700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f6cf0066c40 0x7f6cf014fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:18.472 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf6679700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6cf0150010 0x7f6cf00bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:18.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf6679700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf00634d0 con 0x7f6cf014d050 2023-10-18T18:02:18.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf6679700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf0066890 con 0x7f6cf0150010 2023-10-18T18:02:18.473 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf6679700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf00bc6e0 con 0x7f6cf0066c40 2023-10-18T18:02:18.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf5677700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f6cf014d050 0x7f6cf0062dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:42466/0 (socket says 172.21.15.72:42466) 2023-10-18T18:02:18.474 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf5677700 1 -- 172.21.15.72:0/574708746 learned_addr learned my addr 172.21.15.72:0/574708746 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf4e76700 1 --2- 172.21.15.72:0/574708746 >> v2:172.21.15.179:3300/0 conn(0x7f6cf0066c40 0x7f6cf014fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.465+0000 7f6cf5e78700 1 --2- 172.21.15.72:0/574708746 >> v2:172.21.15.72:3300/0 conn(0x7f6cf0150010 0x7f6cf00bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707278235 0 0) 0x7f6cf00634d0 con 0x7f6cf014d050 2023-10-18T18:02:18.478 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cdc0033f0 con 0x7f6cf014d050 2023-10-18T18:02:18.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 <== mon.0 v1:172.21.15.72:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138887066 0 0) 0x7f6cdc0033f0 con 0x7f6cf014d050 2023-10-18T18:02:18.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 >> v2:172.21.15.179:3300/0 conn(0x7f6cf0066c40 msgr2=0x7f6cf014fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:18.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 --2- 172.21.15.72:0/574708746 >> v2:172.21.15.179:3300/0 conn(0x7f6cf0066c40 0x7f6cf014fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 >> v2:172.21.15.72:3300/0 conn(0x7f6cf0150010 msgr2=0x7f6cf00bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:18.479 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 --2- 172.21.15.72:0/574708746 >> v2:172.21.15.72:3300/0 conn(0x7f6cf0150010 0x7f6cf00bd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 --> v1:172.21.15.72:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf0150620 con 0x7f6cf014d050 2023-10-18T18:02:18.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 <== mon.0 v1:172.21.15.72:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1723103133 0 0) 0x7f6cec001470 con 0x7f6cf014d050 2023-10-18T18:02:18.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 >> v1:172.21.15.72:6789/0 conn(0x7f6cf014d050 legacy=0x7f6cf0062dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:18.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6cdc003910 0x7f6cdc003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:18.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cdc004690 0x7f6cdc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:18.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cdc005000 0x7f6cdc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:18.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0066890 con 0x7f6cdc003910 2023-10-18T18:02:18.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc00be80 con 0x7f6cdc004690 2023-10-18T18:02:18.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc00c040 con 0x7f6cdc005000 2023-10-18T18:02:18.481 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf5e78700 1 --2- 172.21.15.72:0/574708746 >> v2:172.21.15.72:3300/0 conn(0x7f6cf0150010 0x7f6cf00bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:02:18.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf4e76700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cdc004690 0x7f6cdc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:18.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf5e78700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cdc005000 0x7f6cdc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:18.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf5677700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6cdc003910 0x7f6cdc003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:18.482 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf4e76700 1 -- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cdc005000 msgr2=0x7f6cdc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:18.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf4e76700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cdc005000 0x7f6cdc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf4e76700 1 -- 172.21.15.72:0/574708746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6cdc003910 msgr2=0x7f6cdc003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:18.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf4e76700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6cdc003910 0x7f6cdc003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf4e76700 1 -- 172.21.15.72:0/574708746 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0033f0 con 0x7f6cdc004690 2023-10-18T18:02:18.483 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf5e78700 1 -- 172.21.15.72:0/574708746 reap_dead start 2023-10-18T18:02:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6cf4e76700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cdc004690 0x7f6cdc004b10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f6ce0006d80 tx=0x7f6ce0009060).ready entity=mon.1 client_cookie=f796b048adc82034 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0009430 con 0x7f6cdc004690 2023-10-18T18:02:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce000e640 con 0x7f6cdc004690 2023-10-18T18:02:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.469+0000 7f6ce7fff700 1 -- 172.21.15.72:0/574708746 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000e9c0 con 0x7f6cdc004690 2023-10-18T18:02:18.484 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.473+0000 7f6cf6679700 1 -- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cdc004690 msgr2=0x7f6cdc004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:18.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.473+0000 7f6cf6679700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cdc004690 0x7f6cdc004b10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f6ce0006d80 tx=0x7f6ce0009060).stop 2023-10-18T18:02:18.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.473+0000 7f6cf6679700 1 -- 172.21.15.72:0/574708746 shutdown_connections 2023-10-18T18:02:18.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.473+0000 7f6cf6679700 1 --2- 172.21.15.72:0/574708746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cdc004690 0x7f6cdc004b10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.473+0000 7f6cf6679700 1 -- 172.21.15.72:0/574708746 >> 172.21.15.72:0/574708746 conn(0x7f6cf00c2780 msgr2=0x7f6cf00c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:18.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.473+0000 7f6cf6679700 1 -- 172.21.15.72:0/574708746 shutdown_connections 2023-10-18T18:02:18.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.473+0000 7f6cf6679700 1 -- 172.21.15.72:0/574708746 wait complete. 2023-10-18T18:02:18.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.473+0000 7f6cf6679700 1 Processor -- start 2023-10-18T18:02:18.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.473+0000 7f6cf6679700 1 -- start start 2023-10-18T18:02:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf6679700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6cdc004690 0x7f6cf0160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf6679700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cf00c0ac0 0x7f6cf00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf6679700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cf00c0f20 0x7f6cf00b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf6679700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00b3f10 con 0x7f6cdc004690 2023-10-18T18:02:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf6679700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00b5860 con 0x7f6cf00c0f20 2023-10-18T18:02:18.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf6679700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0160b20 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf5677700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6cdc004690 0x7f6cf0160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:18.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf4e76700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cf00c0ac0 0x7f6cf00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:18.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf4e76700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cf00c0ac0 0x7f6cf00b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:37784/0 (socket says 172.21.15.72:37784) 2023-10-18T18:02:18.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf4e76700 1 -- 172.21.15.72:0/2000431486 learned_addr learned my addr 172.21.15.72:0/2000431486 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf5e78700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cf00c0f20 0x7f6cf00b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf4e76700 1 -- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cf00c0f20 msgr2=0x7f6cf00b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf4e76700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cf00c0f20 0x7f6cf00b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf4e76700 1 -- 172.21.15.72:0/2000431486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6cdc004690 msgr2=0x7f6cf0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:18.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf4e76700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6cdc004690 0x7f6cf0160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf4e76700 1 -- 172.21.15.72:0/2000431486 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00b4d10 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf4e76700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cf00c0ac0 0x7f6cf00b4330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f6ce000e4b0 tx=0x7f6ce001a000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6ce57fa700 1 -- 172.21.15.72:0/2000431486 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0009ec0 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6ce57fa700 1 -- 172.21.15.72:0/2000431486 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce000ed30 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf0069470 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6ce57fa700 1 -- 172.21.15.72:0/2000431486 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000eea0 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.477+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf0068c30 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.481+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf00b4050 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.481+0000 7f6ce57fa700 1 -- 172.21.15.72:0/2000431486 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f6ce0043b40 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.481+0000 7f6ce57fa700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6cd405a510 0x7f6cd405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.481+0000 7f6ce57fa700 1 -- 172.21.15.72:0/2000431486 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f6ce0075c90 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.481+0000 7f6cf5677700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6cd405a510 0x7f6cd405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:18.494 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.485+0000 7f6cf5677700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6cd405a510 0x7f6cd405c9d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f6cec0015a0 tx=0x7f6cec007040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:18.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.493+0000 7f6ce57fa700 1 -- 172.21.15.72:0/2000431486 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f6ce004cc90 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.685+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6cf00b4050 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.685+0000 7f6ce57fa700 1 -- 172.21.15.72:0/2000431486 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f6cf00b4050 con 0x7f6cf00c0ac0 2023-10-18T18:02:18.692 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:18.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6cd405a510 msgr2=0x7f6cd405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:18.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6cd405a510 0x7f6cd405c9d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f6cec0015a0 tx=0x7f6cec007040).stop 2023-10-18T18:02:18.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cf00c0ac0 msgr2=0x7f6cf00b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:18.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cf00c0ac0 0x7f6cf00b4330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f6ce000e4b0 tx=0x7f6ce001a000).stop 2023-10-18T18:02:18.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 shutdown_connections 2023-10-18T18:02:18.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f6cd405a510 0x7f6cd405c9d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6cf00c0f20 0x7f6cf00b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6cf00c0ac0 0x7f6cf00b4330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 --2- 172.21.15.72:0/2000431486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6cdc004690 0x7f6cf0160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:18.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 >> 172.21.15.72:0/2000431486 conn(0x7f6cf00c2780 msgr2=0x7f6cf0155250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:18.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 shutdown_connections 2023-10-18T18:02:18.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:18.689+0000 7f6cf6679700 1 -- 172.21.15.72:0/2000431486 wait complete. 2023-10-18T18:02:18.719 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:21.719 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:21.877 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.873+0000 7f43caf07700 1 Processor -- start 2023-10-18T18:02:21.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.873+0000 7f43caf07700 1 -- start start 2023-10-18T18:02:21.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43caf07700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f43c40be300 0x7f43c40bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:21.882 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43caf07700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f43c40686b0 0x7f43c4068af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:21.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43caf07700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c40bd1a0 con 0x7f43c40bdf20 2023-10-18T18:02:21.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43caf07700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c4069e70 con 0x7f43c40686b0 2023-10-18T18:02:21.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43caf07700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c40bd480 con 0x7f43c40be300 2023-10-18T18:02:21.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43c9f05700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f43c40bdf20 0x7f43c40c2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:45878/0 (socket says 172.21.15.72:45878) 2023-10-18T18:02:21.883 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43c9f05700 1 -- 172.21.15.72:0/3395612451 learned_addr learned my addr 172.21.15.72:0/3395612451 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:21.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43ca706700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f43c40686b0 0x7f43c4068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:21.884 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43c9704700 1 --2- 172.21.15.72:0/3395612451 >> v2:172.21.15.179:3300/0 conn(0x7f43c40be300 0x7f43c40bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:21.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43c8f03700 1 -- 172.21.15.72:0/3395612451 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721528756 0 0) 0x7f43c40bd1a0 con 0x7f43c40bdf20 2023-10-18T18:02:21.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43c8f03700 1 -- 172.21.15.72:0/3395612451 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b00033f0 con 0x7f43c40bdf20 2023-10-18T18:02:21.885 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43ca706700 1 -- 172.21.15.72:0/3395612451 >> v2:172.21.15.179:3300/0 conn(0x7f43c40be300 msgr2=0x7f43c40bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:21.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43ca706700 1 --2- 172.21.15.72:0/3395612451 >> v2:172.21.15.179:3300/0 conn(0x7f43c40be300 0x7f43c40bc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:21.886 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43ca706700 1 -- 172.21.15.72:0/3395612451 >> v1:172.21.15.179:6789/0 conn(0x7f43c40bdf20 legacy=0x7f43c40c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:21.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43ca706700 1 -- 172.21.15.72:0/3395612451 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40bd760 con 0x7f43c40686b0 2023-10-18T18:02:21.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43c9704700 1 --2- 172.21.15.72:0/3395612451 >> v2:172.21.15.179:3300/0 conn(0x7f43c40be300 0x7f43c40bc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:21.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43ca706700 1 --2- 172.21.15.72:0/3395612451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40686b0 0x7f43c4068af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f43c40668e0 tx=0x7f43bc009780).ready entity=mon.0 client_cookie=6e909877f3d33458 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:21.887 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43c8f03700 1 -- 172.21.15.72:0/3395612451 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00e070 con 0x7f43c40686b0 2023-10-18T18:02:21.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43c8f03700 1 -- 172.21.15.72:0/3395612451 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc00bab0 con 0x7f43c40686b0 2023-10-18T18:02:21.888 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.877+0000 7f43c8f03700 1 -- 172.21.15.72:0/3395612451 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00be30 con 0x7f43c40686b0 2023-10-18T18:02:21.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 -- 172.21.15.72:0/3395612451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40686b0 msgr2=0x7f43c4068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:21.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 --2- 172.21.15.72:0/3395612451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40686b0 0x7f43c4068af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f43c40668e0 tx=0x7f43bc009780).stop 2023-10-18T18:02:21.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 -- 172.21.15.72:0/3395612451 shutdown_connections 2023-10-18T18:02:21.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 --2- 172.21.15.72:0/3395612451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40686b0 0x7f43c4068af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:21.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 --2- 172.21.15.72:0/3395612451 >> v2:172.21.15.179:3300/0 conn(0x7f43c40be300 0x7f43c40bc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:21.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 -- 172.21.15.72:0/3395612451 >> 172.21.15.72:0/3395612451 conn(0x7f43c4062ba0 msgr2=0x7f43c40c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:21.894 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 -- 172.21.15.72:0/3395612451 shutdown_connections 2023-10-18T18:02:21.894 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 -- 172.21.15.72:0/3395612451 wait complete. 2023-10-18T18:02:21.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 Processor -- start 2023-10-18T18:02:21.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.881+0000 7f43caf07700 1 -- start start 2023-10-18T18:02:21.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43caf07700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43c40686b0 0x7f43c4150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:21.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43caf07700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c40bdf20 0x7f43c40a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:21.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43caf07700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40be300 0x7f43c4153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:21.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43caf07700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c4150c70 con 0x7f43c40be300 2023-10-18T18:02:21.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43caf07700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c414fc60 con 0x7f43c40bdf20 2023-10-18T18:02:21.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43caf07700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c41507e0 con 0x7f43c40686b0 2023-10-18T18:02:21.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43c9704700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c40bdf20 0x7f43c40a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:21.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43c9704700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c40bdf20 0x7f43c40a6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:51738/0 (socket says 172.21.15.72:51738) 2023-10-18T18:02:21.897 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43c9704700 1 -- 172.21.15.72:0/1762022176 learned_addr learned my addr 172.21.15.72:0/1762022176 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:21.897 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43ca706700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40be300 0x7f43c4153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:21.897 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43c9f05700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43c40686b0 0x7f43c4150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:21.897 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43ca706700 1 -- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43c40686b0 msgr2=0x7f43c4150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:21.897 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43ca706700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43c40686b0 0x7f43c4150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:21.898 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43ca706700 1 -- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c40bdf20 msgr2=0x7f43c40a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:21.898 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43ca706700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c40bdf20 0x7f43c40a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:21.898 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43ca706700 1 -- 172.21.15.72:0/1762022176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b40092a0 con 0x7f43c40be300 2023-10-18T18:02:21.898 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43c9704700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c40bdf20 0x7f43c40a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:21.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43ca706700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40be300 0x7f43c4153480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f43c4066ea0 tx=0x7f43bc009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:21.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43b9ffb700 1 -- 172.21.15.72:0/1762022176 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc012e70 con 0x7f43c40be300 2023-10-18T18:02:21.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43b9ffb700 1 -- 172.21.15.72:0/1762022176 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc009440 con 0x7f43c40be300 2023-10-18T18:02:21.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc009a20 con 0x7f43c40be300 2023-10-18T18:02:21.899 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43b9ffb700 1 -- 172.21.15.72:0/1762022176 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc01d860 con 0x7f43c40be300 2023-10-18T18:02:21.900 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.885+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c4151370 con 0x7f43c40be300 2023-10-18T18:02:21.900 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.889+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c4150990 con 0x7f43c40be300 2023-10-18T18:02:21.902 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.889+0000 7f43b9ffb700 1 -- 172.21.15.72:0/1762022176 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f43bc01dc10 con 0x7f43c40be300 2023-10-18T18:02:21.902 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.889+0000 7f43b9ffb700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f43a805a540 0x7f43a805ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:21.902 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.889+0000 7f43b9ffb700 1 -- 172.21.15.72:0/1762022176 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f43bc0764b0 con 0x7f43c40be300 2023-10-18T18:02:21.903 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.889+0000 7f43c9f05700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f43a805a540 0x7f43a805ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:21.903 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.893+0000 7f43c9f05700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f43a805a540 0x7f43a805ca00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f43c0005750 tx=0x7f43c0009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:21.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:21.897+0000 7f43b9ffb700 1 -- 172.21.15.72:0/1762022176 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f43c4150990 con 0x7f43c40be300 2023-10-18T18:02:22.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.097+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f43c4150950 con 0x7f43c40be300 2023-10-18T18:02:22.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.097+0000 7f43b9ffb700 1 -- 172.21.15.72:0/1762022176 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f43bc049a00 con 0x7f43c40be300 2023-10-18T18:02:22.103 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:22.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f43a805a540 msgr2=0x7f43a805ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:22.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f43a805a540 0x7f43a805ca00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f43c0005750 tx=0x7f43c0009040).stop 2023-10-18T18:02:22.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40be300 msgr2=0x7f43c4153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:22.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40be300 0x7f43c4153480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f43c4066ea0 tx=0x7f43bc009780).stop 2023-10-18T18:02:22.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 shutdown_connections 2023-10-18T18:02:22.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f43a805a540 0x7f43a805ca00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:22.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43c40be300 0x7f43c4153480 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:22.107 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c40bdf20 0x7f43c40a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:22.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 --2- 172.21.15.72:0/1762022176 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f43c40686b0 0x7f43c4150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:22.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 >> 172.21.15.72:0/1762022176 conn(0x7f43c4062ba0 msgr2=0x7f43c414cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:22.108 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 shutdown_connections 2023-10-18T18:02:22.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:22.101+0000 7f43caf07700 1 -- 172.21.15.72:0/1762022176 wait complete. 2023-10-18T18:02:22.123 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:23.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~233s 2023-10-18T18:02:23.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~178s 2023-10-18T18:02:23.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~133s 2023-10-18T18:02:23.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~158s 2023-10-18T18:02:23.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~145s 2023-10-18T18:02:23.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~192s 2023-10-18T18:02:25.125 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:25.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.285+0000 7f3dbe146700 1 Processor -- start 2023-10-18T18:02:25.289 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.285+0000 7f3dbe146700 1 -- start start 2023-10-18T18:02:25.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3db8067ff0 0x7f3db80683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:25.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db80666b0 0x7f3db80bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:25.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db8065cd0 con 0x7f3db80bde20 2023-10-18T18:02:25.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db8066160 con 0x7f3db8067ff0 2023-10-18T18:02:25.293 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8154750 con 0x7f3db80666b0 2023-10-18T18:02:25.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db77fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3db8067ff0 0x7f3db80683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:25.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3daeffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db80666b0 0x7f3db80bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:25.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db7fff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f3db80bde20 0x7f3db8151410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:45882/0 (socket says 172.21.15.72:45882) 2023-10-18T18:02:25.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db7fff700 1 -- 172.21.15.72:0/1703624906 learned_addr learned my addr 172.21.15.72:0/1703624906 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:25.295 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db6ffd700 1 -- 172.21.15.72:0/1703624906 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307426139 0 0) 0x7f3db8065cd0 con 0x7f3db80bde20 2023-10-18T18:02:25.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db6ffd700 1 -- 172.21.15.72:0/1703624906 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da40033f0 con 0x7f3db80bde20 2023-10-18T18:02:25.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db77fe700 1 -- 172.21.15.72:0/1703624906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db80666b0 msgr2=0x7f3db80bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:25.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db77fe700 1 --2- 172.21.15.72:0/1703624906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db80666b0 0x7f3db80bd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:25.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db77fe700 1 -- 172.21.15.72:0/1703624906 >> v1:172.21.15.179:6789/0 conn(0x7f3db80bde20 legacy=0x7f3db8151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:25.296 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db77fe700 1 -- 172.21.15.72:0/1703624906 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db8065100 con 0x7f3db8067ff0 2023-10-18T18:02:25.297 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db77fe700 1 --2- 172.21.15.72:0/1703624906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db8067ff0 0x7f3db80683d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f3da8009760 tx=0x7f3da8009060).ready entity=mon.0 client_cookie=58db541ceb728920 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:25.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db6ffd700 1 -- 172.21.15.72:0/1703624906 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da800d4a0 con 0x7f3db8067ff0 2023-10-18T18:02:25.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db6ffd700 1 -- 172.21.15.72:0/1703624906 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da800d600 con 0x7f3db8067ff0 2023-10-18T18:02:25.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3db6ffd700 1 -- 172.21.15.72:0/1703624906 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da800d980 con 0x7f3db8067ff0 2023-10-18T18:02:25.298 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 -- 172.21.15.72:0/1703624906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db8067ff0 msgr2=0x7f3db80683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:25.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 --2- 172.21.15.72:0/1703624906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db8067ff0 0x7f3db80683d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f3da8009760 tx=0x7f3da8009060).stop 2023-10-18T18:02:25.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 -- 172.21.15.72:0/1703624906 shutdown_connections 2023-10-18T18:02:25.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 --2- 172.21.15.72:0/1703624906 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db80666b0 0x7f3db80bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:25.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 --2- 172.21.15.72:0/1703624906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db8067ff0 0x7f3db80683d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:25.299 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 -- 172.21.15.72:0/1703624906 >> 172.21.15.72:0/1703624906 conn(0x7f3db80620d0 msgr2=0x7f3db80624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:25.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.289+0000 7f3dbe146700 1 -- 172.21.15.72:0/1703624906 shutdown_connections 2023-10-18T18:02:25.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.293+0000 7f3dbe146700 1 -- 172.21.15.72:0/1703624906 wait complete. 2023-10-18T18:02:25.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.293+0000 7f3dbe146700 1 Processor -- start 2023-10-18T18:02:25.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.293+0000 7f3dbe146700 1 -- start start 2023-10-18T18:02:25.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3dbe146700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80666b0 0x7f3db8168440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:25.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3dbe146700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db8067ff0 0x7f3db80b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:25.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3dbe146700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db80bde20 0x7f3db80be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:25.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3dbe146700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80b7e40 con 0x7f3db80bde20 2023-10-18T18:02:25.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3dbe146700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8168930 con 0x7f3db80666b0 2023-10-18T18:02:25.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3dbe146700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80b5350 con 0x7f3db8067ff0 2023-10-18T18:02:25.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db7fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db80bde20 0x7f3db80be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:25.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80666b0 0x7f3db8168440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:25.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3daeffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db8067ff0 0x7f3db80b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:25.303 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db7fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db80bde20 0x7f3db80be9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57186/0 (socket says 172.21.15.72:57186) 2023-10-18T18:02:25.304 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db7fff700 1 -- 172.21.15.72:0/3143875827 learned_addr learned my addr 172.21.15.72:0/3143875827 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:25.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db7fff700 1 -- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db8067ff0 msgr2=0x7f3db80b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:25.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db7fff700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db8067ff0 0x7f3db80b8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:25.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db7fff700 1 -- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80666b0 msgr2=0x7f3db8168440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:25.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db7fff700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80666b0 0x7f3db8168440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:25.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db7fff700 1 -- 172.21.15.72:0/3143875827 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80092a0 con 0x7f3db80bde20 2023-10-18T18:02:25.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3db7fff700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db80bde20 0x7f3db80be9a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f3db000c810 tx=0x7f3db0001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:25.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3daffff700 1 -- 172.21.15.72:0/3143875827 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0005d30 con 0x7f3db80bde20 2023-10-18T18:02:25.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3daffff700 1 -- 172.21.15.72:0/3143875827 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db0005e90 con 0x7f3db80bde20 2023-10-18T18:02:25.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3daffff700 1 -- 172.21.15.72:0/3143875827 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0007320 con 0x7f3db80bde20 2023-10-18T18:02:25.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db80b71c0 con 0x7f3db80bde20 2023-10-18T18:02:25.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.297+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db80b7780 con 0x7f3db80bde20 2023-10-18T18:02:25.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.301+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db80b7f80 con 0x7f3db80bde20 2023-10-18T18:02:25.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.301+0000 7f3daffff700 1 -- 172.21.15.72:0/3143875827 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3db0007710 con 0x7f3db80bde20 2023-10-18T18:02:25.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.301+0000 7f3daffff700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3d9805a510 0x7f3d9805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:25.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.301+0000 7f3daffff700 1 -- 172.21.15.72:0/3143875827 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f3db0074890 con 0x7f3db80bde20 2023-10-18T18:02:25.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.301+0000 7f3db77fe700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3d9805a510 0x7f3d9805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:25.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.301+0000 7f3db77fe700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3d9805a510 0x7f3d9805c9d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f3da800d260 tx=0x7f3da8016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:25.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.309+0000 7f3daffff700 1 -- 172.21.15.72:0/3143875827 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f3db0047d80 con 0x7f3db80bde20 2023-10-18T18:02:25.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.501+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3db806c190 con 0x7f3db80bde20 2023-10-18T18:02:25.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.501+0000 7f3daffff700 1 -- 172.21.15.72:0/3143875827 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f3db004b3a0 con 0x7f3db80bde20 2023-10-18T18:02:25.508 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:25.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3d9805a510 msgr2=0x7f3d9805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:25.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3d9805a510 0x7f3d9805c9d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f3da800d260 tx=0x7f3da8016000).stop 2023-10-18T18:02:25.510 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db80bde20 msgr2=0x7f3db80be9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:25.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db80bde20 0x7f3db80be9a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f3db000c810 tx=0x7f3db0001650).stop 2023-10-18T18:02:25.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 shutdown_connections 2023-10-18T18:02:25.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3d9805a510 0x7f3d9805c9d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:25.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3db80bde20 0x7f3db80be9a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:25.511 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3db8067ff0 0x7f3db80b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:25.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 --2- 172.21.15.72:0/3143875827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3db80666b0 0x7f3db8168440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:25.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 >> 172.21.15.72:0/3143875827 conn(0x7f3db80620d0 msgr2=0x7f3db8154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:25.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 shutdown_connections 2023-10-18T18:02:25.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:25.505+0000 7f3dbe146700 1 -- 172.21.15.72:0/3143875827 wait complete. 2023-10-18T18:02:25.530 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:26.910 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:02:26.918 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:02:28.531 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:28.691 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.684+0000 7f626bc1b700 1 Processor -- start 2023-10-18T18:02:28.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.684+0000 7f626bc1b700 1 -- start start 2023-10-18T18:02:28.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f626bc1b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6264067ff0 0x7f62640683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:28.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f626bc1b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62640666b0 0x7f62640bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:28.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f626bc1b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6264065cd0 con 0x7f62640bde20 2023-10-18T18:02:28.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f626bc1b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6264066160 con 0x7f6264067ff0 2023-10-18T18:02:28.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f626bc1b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6264154750 con 0x7f62640666b0 2023-10-18T18:02:28.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62699b7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f6264067ff0 0x7f62640683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:28.695 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f626a1b8700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f62640bde20 0x7f6264151410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:45898/0 (socket says 172.21.15.72:45898) 2023-10-18T18:02:28.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f626a1b8700 1 -- 172.21.15.72:0/1082636581 learned_addr learned my addr 172.21.15.72:0/1082636581 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:28.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62691b6700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62640666b0 0x7f62640bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:28.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62689b5700 1 -- 172.21.15.72:0/1082636581 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577319381 0 0) 0x7f6264065cd0 con 0x7f62640bde20 2023-10-18T18:02:28.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62689b5700 1 -- 172.21.15.72:0/1082636581 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62500033f0 con 0x7f62640bde20 2023-10-18T18:02:28.696 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62691b6700 1 -- 172.21.15.72:0/1082636581 >> v2:172.21.15.72:3300/0 conn(0x7f6264067ff0 msgr2=0x7f62640683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:28.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62691b6700 1 --2- 172.21.15.72:0/1082636581 >> v2:172.21.15.72:3300/0 conn(0x7f6264067ff0 0x7f62640683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:28.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62691b6700 1 -- 172.21.15.72:0/1082636581 >> v1:172.21.15.179:6789/0 conn(0x7f62640bde20 legacy=0x7f6264151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:28.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62691b6700 1 -- 172.21.15.72:0/1082636581 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6264065100 con 0x7f62640666b0 2023-10-18T18:02:28.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62699b7700 1 --2- 172.21.15.72:0/1082636581 >> v2:172.21.15.72:3300/0 conn(0x7f6264067ff0 0x7f62640683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:28.697 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62691b6700 1 --2- 172.21.15.72:0/1082636581 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62640666b0 0x7f62640bd930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f6254009760 tx=0x7f6254009060).ready entity=mon.2 client_cookie=43c623befb925e54 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:28.698 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62689b5700 1 -- 172.21.15.72:0/1082636581 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625400d4a0 con 0x7f62640666b0 2023-10-18T18:02:28.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62689b5700 1 -- 172.21.15.72:0/1082636581 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625400d600 con 0x7f62640666b0 2023-10-18T18:02:28.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.688+0000 7f62689b5700 1 -- 172.21.15.72:0/1082636581 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625400d980 con 0x7f62640666b0 2023-10-18T18:02:28.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 -- 172.21.15.72:0/1082636581 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62640666b0 msgr2=0x7f62640bd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:28.699 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 --2- 172.21.15.72:0/1082636581 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62640666b0 0x7f62640bd930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f6254009760 tx=0x7f6254009060).stop 2023-10-18T18:02:28.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 -- 172.21.15.72:0/1082636581 shutdown_connections 2023-10-18T18:02:28.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 --2- 172.21.15.72:0/1082636581 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f62640666b0 0x7f62640bd930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:28.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 --2- 172.21.15.72:0/1082636581 >> v2:172.21.15.72:3300/0 conn(0x7f6264067ff0 0x7f62640683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:28.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 -- 172.21.15.72:0/1082636581 >> 172.21.15.72:0/1082636581 conn(0x7f62640620d0 msgr2=0x7f62640624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:28.700 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 -- 172.21.15.72:0/1082636581 shutdown_connections 2023-10-18T18:02:28.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 -- 172.21.15.72:0/1082636581 wait complete. 2023-10-18T18:02:28.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 Processor -- start 2023-10-18T18:02:28.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.692+0000 7f626bc1b700 1 -- start start 2023-10-18T18:02:28.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f626bc1b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62640666b0 0x7f6264168630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:28.702 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f626bc1b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6264067ff0 0x7f62640b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:28.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f626bc1b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62640bde20 0x7f6264153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:28.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f626bc1b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6264154190 con 0x7f62640666b0 2023-10-18T18:02:28.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f626bc1b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640b7030 con 0x7f62640bde20 2023-10-18T18:02:28.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f626bc1b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62640b72d0 con 0x7f6264067ff0 2023-10-18T18:02:28.703 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62699b7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62640666b0 0x7f6264168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:28.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62699b7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62640666b0 0x7f6264168630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:57206/0 (socket says 172.21.15.72:57206) 2023-10-18T18:02:28.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62699b7700 1 -- 172.21.15.72:0/156771163 learned_addr learned my addr 172.21.15.72:0/156771163 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:28.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62691b6700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6264067ff0 0x7f62640b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:28.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f626a1b8700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62640bde20 0x7f6264153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:28.705 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62699b7700 1 -- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6264067ff0 msgr2=0x7f62640b86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:28.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62699b7700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6264067ff0 0x7f62640b86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:28.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62699b7700 1 -- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62640bde20 msgr2=0x7f6264153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:28.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62699b7700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62640bde20 0x7f6264153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:28.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62699b7700 1 -- 172.21.15.72:0/156771163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62600092a0 con 0x7f62640666b0 2023-10-18T18:02:28.706 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f62699b7700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62640666b0 0x7f6264168630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f626000e980 tx=0x7f6260009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:28.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f6259ffb700 1 -- 172.21.15.72:0/156771163 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260012600 con 0x7f62640666b0 2023-10-18T18:02:28.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f6259ffb700 1 -- 172.21.15.72:0/156771163 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6260012760 con 0x7f62640666b0 2023-10-18T18:02:28.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f6259ffb700 1 -- 172.21.15.72:0/156771163 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260012ae0 con 0x7f62640666b0 2023-10-18T18:02:28.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62540092a0 con 0x7f62640666b0 2023-10-18T18:02:28.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.696+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62640b7890 con 0x7f62640666b0 2023-10-18T18:02:28.707 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.700+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626406c190 con 0x7f62640666b0 2023-10-18T18:02:28.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.700+0000 7f6259ffb700 1 -- 172.21.15.72:0/156771163 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f626001b400 con 0x7f62640666b0 2023-10-18T18:02:28.708 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.700+0000 7f6259ffb700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f624805a510 0x7f624805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:28.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.700+0000 7f6259ffb700 1 -- 172.21.15.72:0/156771163 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f6260074ec0 con 0x7f62640666b0 2023-10-18T18:02:28.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.700+0000 7f62691b6700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f624805a510 0x7f624805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:28.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.704+0000 7f62691b6700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f624805a510 0x7f624805c9d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f625400d260 tx=0x7f6254016000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:28.712 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.704+0000 7f6259ffb700 1 -- 172.21.15.72:0/156771163 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f62600483b0 con 0x7f62640666b0 2023-10-18T18:02:28.900 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.892+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6264063cf0 con 0x7f62640666b0 2023-10-18T18:02:28.901 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f6259ffb700 1 -- 172.21.15.72:0/156771163 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f626004b9d0 con 0x7f62640666b0 2023-10-18T18:02:28.901 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f624805a510 msgr2=0x7f624805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f624805a510 0x7f624805c9d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f625400d260 tx=0x7f6254016000).stop 2023-10-18T18:02:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62640666b0 msgr2=0x7f6264168630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:28.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62640666b0 0x7f6264168630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f626000e980 tx=0x7f6260009060).stop 2023-10-18T18:02:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 shutdown_connections 2023-10-18T18:02:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f624805a510 0x7f624805c9d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62640bde20 0x7f6264153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6264067ff0 0x7f62640b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:28.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 --2- 172.21.15.72:0/156771163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f62640666b0 0x7f6264168630 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:28.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 >> 172.21.15.72:0/156771163 conn(0x7f62640620d0 msgr2=0x7f6264156200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:28.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 shutdown_connections 2023-10-18T18:02:28.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:28.896+0000 7f626bc1b700 1 -- 172.21.15.72:0/156771163 wait complete. 2023-10-18T18:02:28.923 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:29.979 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~239s 2023-10-18T18:02:29.979 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~185s 2023-10-18T18:02:29.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~139s 2023-10-18T18:02:29.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~165s 2023-10-18T18:02:29.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~152s 2023-10-18T18:02:29.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~198s 2023-10-18T18:02:31.924 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:32.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.076+0000 7f678f0e4700 1 Processor -- start 2023-10-18T18:02:32.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.076+0000 7f678f0e4700 1 -- start start 2023-10-18T18:02:32.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678f0e4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880be300 0x7f67880bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:32.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678f0e4700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f67880686b0 0x7f6788068b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:32.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678f0e4700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67880bd1a0 con 0x7f67880bdf20 2023-10-18T18:02:32.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678f0e4700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6788069e70 con 0x7f67880686b0 2023-10-18T18:02:32.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678f0e4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67880bd480 con 0x7f67880be300 2023-10-18T18:02:32.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678e8e3700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f67880686b0 0x7f6788068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:32.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678e0e2700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f67880bdf20 0x7f67880c2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:41070/0 (socket says 172.21.15.72:41070) 2023-10-18T18:02:32.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678e0e2700 1 -- 172.21.15.72:0/1940458725 learned_addr learned my addr 172.21.15.72:0/1940458725 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:32.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678d8e1700 1 --2- 172.21.15.72:0/1940458725 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880be300 0x7f67880bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:32.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678e8e3700 1 -- 172.21.15.72:0/1940458725 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880be300 msgr2=0x7f67880bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:32.089 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678e8e3700 1 --2- 172.21.15.72:0/1940458725 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880be300 0x7f67880bc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:32.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678e8e3700 1 -- 172.21.15.72:0/1940458725 >> v1:172.21.15.179:6789/0 conn(0x7f67880bdf20 legacy=0x7f67880c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:32.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678e8e3700 1 -- 172.21.15.72:0/1940458725 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67880bd760 con 0x7f67880686b0 2023-10-18T18:02:32.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678e8e3700 1 --2- 172.21.15.72:0/1940458725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 0x7f6788068b20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f678000d0d0 tx=0x7f678000b990).ready entity=mon.0 client_cookie=22b11096a2ac75e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:32.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678d0e0700 1 -- 172.21.15.72:0/1940458725 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780010070 con 0x7f67880686b0 2023-10-18T18:02:32.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678d0e0700 1 -- 172.21.15.72:0/1940458725 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678000ddf0 con 0x7f67880686b0 2023-10-18T18:02:32.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.080+0000 7f678d0e0700 1 -- 172.21.15.72:0/1940458725 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678000b2e0 con 0x7f67880686b0 2023-10-18T18:02:32.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 -- 172.21.15.72:0/1940458725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 msgr2=0x7f6788068b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:32.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 --2- 172.21.15.72:0/1940458725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 0x7f6788068b20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f678000d0d0 tx=0x7f678000b990).stop 2023-10-18T18:02:32.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 -- 172.21.15.72:0/1940458725 shutdown_connections 2023-10-18T18:02:32.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 --2- 172.21.15.72:0/1940458725 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 0x7f6788068b20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:32.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 --2- 172.21.15.72:0/1940458725 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880be300 0x7f67880bc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:32.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 -- 172.21.15.72:0/1940458725 >> 172.21.15.72:0/1940458725 conn(0x7f6788062ba0 msgr2=0x7f67880c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:32.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 -- 172.21.15.72:0/1940458725 shutdown_connections 2023-10-18T18:02:32.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 -- 172.21.15.72:0/1940458725 wait complete. 2023-10-18T18:02:32.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 Processor -- start 2023-10-18T18:02:32.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.084+0000 7f678f0e4700 1 -- start start 2023-10-18T18:02:32.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678f0e4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 0x7f67880b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:32.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678f0e4700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880bdf20 0x7f67880a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:32.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678f0e4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67880be300 0x7f678815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:32.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678f0e4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67880a0980 con 0x7f67880686b0 2023-10-18T18:02:32.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678f0e4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67880b6fe0 con 0x7f67880be300 2023-10-18T18:02:32.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678f0e4700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678815c890 con 0x7f67880bdf20 2023-10-18T18:02:32.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e0e2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 0x7f67880b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:32.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e0e2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 0x7f67880b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43548/0 (socket says 172.21.15.72:43548) 2023-10-18T18:02:32.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e0e2700 1 -- 172.21.15.72:0/1713970775 learned_addr learned my addr 172.21.15.72:0/1713970775 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:32.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e8e3700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67880be300 0x7f678815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:32.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678d8e1700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880bdf20 0x7f67880a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:32.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e0e2700 1 -- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880bdf20 msgr2=0x7f67880a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:32.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e0e2700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880bdf20 0x7f67880a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:32.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e0e2700 1 -- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67880be300 msgr2=0x7f678815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:32.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e0e2700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67880be300 0x7f678815c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:32.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e0e2700 1 -- 172.21.15.72:0/1713970775 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678000bc30 con 0x7f67880686b0 2023-10-18T18:02:32.098 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678e0e2700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 0x7f67880b7470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f678400c4c0 tx=0x7f6784007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:32.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f677e7fc700 1 -- 172.21.15.72:0/1713970775 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840070d0 con 0x7f67880686b0 2023-10-18T18:02:32.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f677e7fc700 1 -- 172.21.15.72:0/1713970775 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6784007230 con 0x7f67880686b0 2023-10-18T18:02:32.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f677e7fc700 1 -- 172.21.15.72:0/1713970775 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840075b0 con 0x7f67880686b0 2023-10-18T18:02:32.099 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67880c1240 con 0x7f67880686b0 2023-10-18T18:02:32.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67880c1800 con 0x7f67880686b0 2023-10-18T18:02:32.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.088+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6788063830 con 0x7f67880686b0 2023-10-18T18:02:32.100 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.092+0000 7f677e7fc700 1 -- 172.21.15.72:0/1713970775 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f67840059b0 con 0x7f67880686b0 2023-10-18T18:02:32.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.092+0000 7f677e7fc700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f676c05a510 0x7f676c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:32.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.092+0000 7f677e7fc700 1 -- 172.21.15.72:0/1713970775 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f6784073e80 con 0x7f67880686b0 2023-10-18T18:02:32.101 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.092+0000 7f678d8e1700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f676c05a510 0x7f676c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:32.102 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.096+0000 7f678d8e1700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f676c05a510 0x7f676c05c9d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f678814f680 tx=0x7f6778008040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:32.106 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.100+0000 7f677e7fc700 1 -- 172.21.15.72:0/1713970775 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f6788063830 con 0x7f67880686b0 2023-10-18T18:02:32.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.296+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6788063830 con 0x7f67880686b0 2023-10-18T18:02:32.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.296+0000 7f677e7fc700 1 -- 172.21.15.72:0/1713970775 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f6788063830 con 0x7f67880686b0 2023-10-18T18:02:32.303 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:32.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f676c05a510 msgr2=0x7f676c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:32.305 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f676c05a510 0x7f676c05c9d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f678814f680 tx=0x7f6778008040).stop 2023-10-18T18:02:32.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 msgr2=0x7f67880b7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:32.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 0x7f67880b7470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f678400c4c0 tx=0x7f6784007de0).stop 2023-10-18T18:02:32.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 shutdown_connections 2023-10-18T18:02:32.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f676c05a510 0x7f676c05c9d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:32.306 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67880be300 0x7f678815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:32.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f67880bdf20 0x7f67880a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:32.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 --2- 172.21.15.72:0/1713970775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67880686b0 0x7f67880b7470 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:32.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 >> 172.21.15.72:0/1713970775 conn(0x7f6788062ba0 msgr2=0x7f67880babf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:32.307 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 shutdown_connections 2023-10-18T18:02:32.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:32.300+0000 7f678f0e4700 1 -- 172.21.15.72:0/1713970775 wait complete. 2023-10-18T18:02:32.324 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:35.325 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:35.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2df347700 1 Processor -- start 2023-10-18T18:02:35.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2df347700 1 -- start start 2023-10-18T18:02:35.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2df347700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d80658a0 0x7fc2d8065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:35.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2df347700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fc2d8069f20 0x7fc2d80c2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:35.488 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2df347700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d806a760 con 0x7fc2d80c2b20 2023-10-18T18:02:35.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2df347700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d8150bf0 con 0x7fc2d8069f20 2023-10-18T18:02:35.489 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2df347700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d806a480 con 0x7fc2d80658a0 2023-10-18T18:02:35.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2deb46700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fc2d80c2b20 0x7fc2d80630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51108/0 (socket says 172.21.15.72:51108) 2023-10-18T18:02:35.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2deb46700 1 -- 172.21.15.72:0/406338276 learned_addr learned my addr 172.21.15.72:0/406338276 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:35.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2de345700 1 --2- 172.21.15.72:0/406338276 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d80658a0 0x7fc2d8065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:35.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.480+0000 7fc2ddb44700 1 --2- 172.21.15.72:0/406338276 >> v2:172.21.15.179:3300/0 conn(0x7fc2d8069f20 0x7fc2d80c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:35.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2de345700 1 -- 172.21.15.72:0/406338276 >> v2:172.21.15.179:3300/0 conn(0x7fc2d8069f20 msgr2=0x7fc2d80c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:35.491 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2de345700 1 --2- 172.21.15.72:0/406338276 >> v2:172.21.15.179:3300/0 conn(0x7fc2d8069f20 0x7fc2d80c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:35.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2dd343700 1 -- 172.21.15.72:0/406338276 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183611710 0 0) 0x7fc2d806a760 con 0x7fc2d80c2b20 2023-10-18T18:02:35.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2de345700 1 -- 172.21.15.72:0/406338276 >> v1:172.21.15.72:6789/0 conn(0x7fc2d80c2b20 legacy=0x7fc2d80630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:35.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2de345700 1 -- 172.21.15.72:0/406338276 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d814fe30 con 0x7fc2d80658a0 2023-10-18T18:02:35.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2de345700 1 --2- 172.21.15.72:0/406338276 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d80658a0 0x7fc2d8065c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fc2d4009760 tx=0x7fc2d4009060).ready entity=mon.2 client_cookie=e42e24ed923ff158 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:35.492 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2dd343700 1 -- 172.21.15.72:0/406338276 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d400b590 con 0x7fc2d80658a0 2023-10-18T18:02:35.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2dd343700 1 -- 172.21.15.72:0/406338276 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d400b6f0 con 0x7fc2d80658a0 2023-10-18T18:02:35.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2dd343700 1 -- 172.21.15.72:0/406338276 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d400ba70 con 0x7fc2d80658a0 2023-10-18T18:02:35.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2df347700 1 -- 172.21.15.72:0/406338276 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d80658a0 msgr2=0x7fc2d8065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:35.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2df347700 1 --2- 172.21.15.72:0/406338276 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d80658a0 0x7fc2d8065c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fc2d4009760 tx=0x7fc2d4009060).stop 2023-10-18T18:02:35.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2df347700 1 -- 172.21.15.72:0/406338276 shutdown_connections 2023-10-18T18:02:35.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2df347700 1 --2- 172.21.15.72:0/406338276 >> v2:172.21.15.179:3300/0 conn(0x7fc2d8069f20 0x7fc2d80c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:35.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2df347700 1 --2- 172.21.15.72:0/406338276 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d80658a0 0x7fc2d8065c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:35.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2df347700 1 -- 172.21.15.72:0/406338276 >> 172.21.15.72:0/406338276 conn(0x7fc2d8062ba0 msgr2=0x7fc2d80c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:35.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2df347700 1 -- 172.21.15.72:0/406338276 shutdown_connections 2023-10-18T18:02:35.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.484+0000 7fc2df347700 1 -- 172.21.15.72:0/406338276 wait complete. 2023-10-18T18:02:35.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2df347700 1 Processor -- start 2023-10-18T18:02:35.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2df347700 1 -- start start 2023-10-18T18:02:35.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2df347700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2d80658a0 0x7fc2d8160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:35.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2df347700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d8069f20 0x7fc2d80b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:35.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2df347700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2d80c2b20 0x7fc2d8156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:35.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2df347700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80b8e30 con 0x7fc2d80658a0 2023-10-18T18:02:35.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2df347700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d8161380 con 0x7fc2d80c2b20 2023-10-18T18:02:35.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2df347700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80a73e0 con 0x7fc2d8069f20 2023-10-18T18:02:35.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2de345700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2d80658a0 0x7fc2d8160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:35.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2ddb44700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d8069f20 0x7fc2d80b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:35.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2de345700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2d80658a0 0x7fc2d8160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43556/0 (socket says 172.21.15.72:43556) 2023-10-18T18:02:35.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2de345700 1 -- 172.21.15.72:0/861358100 learned_addr learned my addr 172.21.15.72:0/861358100 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:35.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2deb46700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2d80c2b20 0x7fc2d8156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:35.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2ddb44700 1 -- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2d80c2b20 msgr2=0x7fc2d8156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:35.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2ddb44700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2d80c2b20 0x7fc2d8156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:35.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2ddb44700 1 -- 172.21.15.72:0/861358100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2d80658a0 msgr2=0x7fc2d8160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:35.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2ddb44700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2d80658a0 0x7fc2d8160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:35.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2ddb44700 1 -- 172.21.15.72:0/861358100 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40092a0 con 0x7fc2d8069f20 2023-10-18T18:02:35.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2de345700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2d80658a0 0x7fc2d8160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:35.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.488+0000 7fc2ddb44700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d8069f20 0x7fc2d80b6a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fc2c800e930 tx=0x7fc2c8009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:35.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.492+0000 7fc2ce7fc700 1 -- 172.21.15.72:0/861358100 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8012600 con 0x7fc2d8069f20 2023-10-18T18:02:35.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.492+0000 7fc2ce7fc700 1 -- 172.21.15.72:0/861358100 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c8012760 con 0x7fc2d8069f20 2023-10-18T18:02:35.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.492+0000 7fc2ce7fc700 1 -- 172.21.15.72:0/861358100 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8012ae0 con 0x7fc2d8069f20 2023-10-18T18:02:35.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.492+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d80a76c0 con 0x7fc2d8069f20 2023-10-18T18:02:35.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.492+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d80a5bf0 con 0x7fc2d8069f20 2023-10-18T18:02:35.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.492+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d8046f00 con 0x7fc2d8069f20 2023-10-18T18:02:35.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.492+0000 7fc2ce7fc700 1 -- 172.21.15.72:0/861358100 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fc2c801b400 con 0x7fc2d8069f20 2023-10-18T18:02:35.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.496+0000 7fc2ce7fc700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc2bc05a510 0x7fc2bc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:35.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.496+0000 7fc2ce7fc700 1 -- 172.21.15.72:0/861358100 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fc2c8074dc0 con 0x7fc2d8069f20 2023-10-18T18:02:35.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.496+0000 7fc2de345700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc2bc05a510 0x7fc2bc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:35.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.496+0000 7fc2de345700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc2bc05a510 0x7fc2bc05c9d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fc2d4009fd0 tx=0x7fc2d4017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:35.512 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.504+0000 7fc2ce7fc700 1 -- 172.21.15.72:0/861358100 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fc2c80482b0 con 0x7fc2d8069f20 2023-10-18T18:02:35.709 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.704+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc2d80661a0 con 0x7fc2d8069f20 2023-10-18T18:02:35.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.704+0000 7fc2ce7fc700 1 -- 172.21.15.72:0/861358100 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fc2c804b8d0 con 0x7fc2d8069f20 2023-10-18T18:02:35.710 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:35.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc2bc05a510 msgr2=0x7fc2bc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:35.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc2bc05a510 0x7fc2bc05c9d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fc2d4009fd0 tx=0x7fc2d4017000).stop 2023-10-18T18:02:35.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d8069f20 msgr2=0x7fc2d80b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:35.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d8069f20 0x7fc2d80b6a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fc2c800e930 tx=0x7fc2c8009060).stop 2023-10-18T18:02:35.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 shutdown_connections 2023-10-18T18:02:35.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fc2bc05a510 0x7fc2bc05c9d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:35.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2d80c2b20 0x7fc2d8156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:35.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fc2d8069f20 0x7fc2d80b6a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:35.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 --2- 172.21.15.72:0/861358100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2d80658a0 0x7fc2d8160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:35.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 >> 172.21.15.72:0/861358100 conn(0x7fc2d8062ba0 msgr2=0x7fc2d80a48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:35.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 shutdown_connections 2023-10-18T18:02:35.717 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:35.708+0000 7fc2df347700 1 -- 172.21.15.72:0/861358100 wait complete. 2023-10-18T18:02:35.737 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:36.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~245s 2023-10-18T18:02:36.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~191s 2023-10-18T18:02:36.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~145s 2023-10-18T18:02:36.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~171s 2023-10-18T18:02:36.292 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~158s 2023-10-18T18:02:36.292 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~204s 2023-10-18T18:02:38.738 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:38.901 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.896+0000 7f34cdc1b700 1 Processor -- start 2023-10-18T18:02:38.902 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.896+0000 7f34cdc1b700 1 -- start start 2023-10-18T18:02:38.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.896+0000 7f34cdc1b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f34c8067ff0 0x7f34c80683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:38.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.896+0000 7f34cdc1b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f34c8065e30 0x7f34c80bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:38.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c8066900 con 0x7f34c80bca50 2023-10-18T18:02:38.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c8068a50 con 0x7f34c8065e30 2023-10-18T18:02:38.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c80659a0 con 0x7f34c8067ff0 2023-10-18T18:02:38.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34bffff700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f34c8065e30 0x7f34c80bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:38.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34c7fff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f34c80bca50 0x7f34c8150890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:51116/0 (socket says 172.21.15.72:51116) 2023-10-18T18:02:38.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34c77fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f34c8067ff0 0x7f34c80683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:38.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34c7fff700 1 -- 172.21.15.72:0/3166122111 learned_addr learned my addr 172.21.15.72:0/3166122111 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:38.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34bffff700 1 -- 172.21.15.72:0/3166122111 >> v2:172.21.15.179:3300/0 conn(0x7f34c8067ff0 msgr2=0x7f34c80683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:38.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34bffff700 1 --2- 172.21.15.72:0/3166122111 >> v2:172.21.15.179:3300/0 conn(0x7f34c8067ff0 0x7f34c80683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:38.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34bffff700 1 -- 172.21.15.72:0/3166122111 >> v1:172.21.15.72:6789/0 conn(0x7f34c80bca50 legacy=0x7f34c8150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:38.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34bffff700 1 -- 172.21.15.72:0/3166122111 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c8066620 con 0x7f34c8065e30 2023-10-18T18:02:38.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34bffff700 1 --2- 172.21.15.72:0/3166122111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c8065e30 0x7f34c80bdc30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f34c805e8c0 tx=0x7f34b0009060).ready entity=mon.0 client_cookie=a9d33d812f8b85ad server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:38.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34c6ffd700 1 -- 172.21.15.72:0/3166122111 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000f510 con 0x7f34c8065e30 2023-10-18T18:02:38.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34c6ffd700 1 -- 172.21.15.72:0/3166122111 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b000f670 con 0x7f34c8065e30 2023-10-18T18:02:38.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34c6ffd700 1 -- 172.21.15.72:0/3166122111 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000f9f0 con 0x7f34c8065e30 2023-10-18T18:02:38.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 -- 172.21.15.72:0/3166122111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c8065e30 msgr2=0x7f34c80bdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:38.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 --2- 172.21.15.72:0/3166122111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c8065e30 0x7f34c80bdc30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f34c805e8c0 tx=0x7f34b0009060).stop 2023-10-18T18:02:38.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 -- 172.21.15.72:0/3166122111 shutdown_connections 2023-10-18T18:02:38.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 --2- 172.21.15.72:0/3166122111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c8065e30 0x7f34c80bdc30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:38.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 --2- 172.21.15.72:0/3166122111 >> v2:172.21.15.179:3300/0 conn(0x7f34c8067ff0 0x7f34c80683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:38.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 -- 172.21.15.72:0/3166122111 >> 172.21.15.72:0/3166122111 conn(0x7f34c80620d0 msgr2=0x7f34c80624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:38.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 -- 172.21.15.72:0/3166122111 shutdown_connections 2023-10-18T18:02:38.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.900+0000 7f34cdc1b700 1 -- 172.21.15.72:0/3166122111 wait complete. 2023-10-18T18:02:38.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.904+0000 7f34cdc1b700 1 Processor -- start 2023-10-18T18:02:38.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.904+0000 7f34cdc1b700 1 -- start start 2023-10-18T18:02:38.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34cdc1b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c8065e30 0x7f34c80c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:38.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34cdc1b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34c8067ff0 0x7f34c80b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:38.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34cdc1b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c80bca50 0x7f34c80c0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:38.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34cdc1b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c8163130 con 0x7f34c80bca50 2023-10-18T18:02:38.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34cdc1b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c814bbb0 con 0x7f34c8065e30 2023-10-18T18:02:38.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34cdc1b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c8163320 con 0x7f34c8067ff0 2023-10-18T18:02:38.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34c77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c8065e30 0x7f34c80c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:38.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34c7fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c80bca50 0x7f34c80c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:38.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34c77fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c8065e30 0x7f34c80c2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:50586/0 (socket says 172.21.15.72:50586) 2023-10-18T18:02:38.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34c77fe700 1 -- 172.21.15.72:0/2996914000 learned_addr learned my addr 172.21.15.72:0/2996914000 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:38.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bffff700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34c8067ff0 0x7f34c80b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:38.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bffff700 1 -- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c8065e30 msgr2=0x7f34c80c2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:38.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bffff700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c8065e30 0x7f34c80c2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:38.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bffff700 1 -- 172.21.15.72:0/2996914000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c80bca50 msgr2=0x7f34c80c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:38.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bffff700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c80bca50 0x7f34c80c0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:38.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bffff700 1 -- 172.21.15.72:0/2996914000 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b00092a0 con 0x7f34c8067ff0 2023-10-18T18:02:38.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34c77fe700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c8065e30 0x7f34c80c2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:38.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34c7fff700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c80bca50 0x7f34c80c0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:38.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bffff700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34c8067ff0 0x7f34c80b4a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f34b0014000 tx=0x7f34b0017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:38.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bf7fe700 1 -- 172.21.15.72:0/2996914000 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000f7e0 con 0x7f34c8067ff0 2023-10-18T18:02:38.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bf7fe700 1 -- 172.21.15.72:0/2996914000 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b000f940 con 0x7f34c8067ff0 2023-10-18T18:02:38.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34bf7fe700 1 -- 172.21.15.72:0/2996914000 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b001b690 con 0x7f34c8067ff0 2023-10-18T18:02:38.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c8163600 con 0x7f34c8067ff0 2023-10-18T18:02:38.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.908+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c8163bc0 con 0x7f34c8067ff0 2023-10-18T18:02:38.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.912+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c806c190 con 0x7f34c8067ff0 2023-10-18T18:02:38.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.912+0000 7f34bf7fe700 1 -- 172.21.15.72:0/2996914000 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f34b001ba40 con 0x7f34c8067ff0 2023-10-18T18:02:38.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.912+0000 7f34bf7fe700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f34a805a510 0x7f34a805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:38.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.912+0000 7f34bf7fe700 1 -- 172.21.15.72:0/2996914000 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f34b00762a0 con 0x7f34c8067ff0 2023-10-18T18:02:38.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.912+0000 7f34c77fe700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f34a805a510 0x7f34a805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:38.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.916+0000 7f34c77fe700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f34a805a510 0x7f34a805c9d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f34b8011310 tx=0x7f34b8009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:38.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:38.924+0000 7f34bf7fe700 1 -- 172.21.15.72:0/2996914000 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f34b00497f0 con 0x7f34c8067ff0 2023-10-18T18:02:39.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.116+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f34c8063cf0 con 0x7f34c8067ff0 2023-10-18T18:02:39.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.116+0000 7f34bf7fe700 1 -- 172.21.15.72:0/2996914000 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f34b004ce10 con 0x7f34c8067ff0 2023-10-18T18:02:39.123 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:39.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f34a805a510 msgr2=0x7f34a805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:39.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f34a805a510 0x7f34a805c9d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f34b8011310 tx=0x7f34b8009060).stop 2023-10-18T18:02:39.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34c8067ff0 msgr2=0x7f34c80b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:39.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34c8067ff0 0x7f34c80b4a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f34b0014000 tx=0x7f34b0017000).stop 2023-10-18T18:02:39.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 shutdown_connections 2023-10-18T18:02:39.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f34a805a510 0x7f34a805c9d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:39.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34c80bca50 0x7f34c80c0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:39.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f34c8067ff0 0x7f34c80b4a10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:39.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 --2- 172.21.15.72:0/2996914000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34c8065e30 0x7f34c80c2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:39.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 >> 172.21.15.72:0/2996914000 conn(0x7f34c80620d0 msgr2=0x7f34c814cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:39.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 shutdown_connections 2023-10-18T18:02:39.127 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:39.120+0000 7f34cdc1b700 1 -- 172.21.15.72:0/2996914000 wait complete. 2023-10-18T18:02:39.144 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:42.145 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:42.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde4aa5b700 1 Processor -- start 2023-10-18T18:02:42.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde4aa5b700 1 -- start start 2023-10-18T18:02:42.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde4aa5b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fde44067ff0 0x7fde440683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:42.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde4aa5b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde440bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:42.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde4aa5b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde44066900 con 0x7fde440bca50 2023-10-18T18:02:42.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde4aa5b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde44068a50 con 0x7fde44067ff0 2023-10-18T18:02:42.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde4aa5b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde440659a0 con 0x7fde44065e30 2023-10-18T18:02:42.323 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde3b7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde440bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:42.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde43fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fde44067ff0 0x7fde440683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:42.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde43fff700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fde44067ff0 0x7fde440683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:48786/0 (socket says 172.21.15.72:48786) 2023-10-18T18:02:42.324 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde43fff700 1 -- 172.21.15.72:0/181955203 learned_addr learned my addr 172.21.15.72:0/181955203 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:42.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde3b7fe700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde440bdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:35928/0 (socket says 172.21.15.72:35928) 2023-10-18T18:02:42.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde43fff700 1 -- 172.21.15.72:0/181955203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 msgr2=0x7fde440bdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:42.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.312+0000 7fde43fff700 1 --2- 172.21.15.72:0/181955203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde440bdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:42.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde43fff700 1 -- 172.21.15.72:0/181955203 >> v1:172.21.15.72:6789/0 conn(0x7fde440bca50 legacy=0x7fde44150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:42.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde43fff700 1 -- 172.21.15.72:0/181955203 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44066620 con 0x7fde44067ff0 2023-10-18T18:02:42.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde43fff700 1 --2- 172.21.15.72:0/181955203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde44067ff0 0x7fde440683d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fde34009830 tx=0x7fde340090f0).ready entity=mon.1 client_cookie=b1769ade7d8b630a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:42.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde437fe700 1 -- 172.21.15.72:0/181955203 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3400b610 con 0x7fde44067ff0 2023-10-18T18:02:42.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde437fe700 1 -- 172.21.15.72:0/181955203 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3400b770 con 0x7fde44067ff0 2023-10-18T18:02:42.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde437fe700 1 -- 172.21.15.72:0/181955203 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3400baf0 con 0x7fde44067ff0 2023-10-18T18:02:42.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde4aa5b700 1 -- 172.21.15.72:0/181955203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde44067ff0 msgr2=0x7fde440683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:42.328 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde4aa5b700 1 --2- 172.21.15.72:0/181955203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde44067ff0 0x7fde440683d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fde34009830 tx=0x7fde340090f0).stop 2023-10-18T18:02:42.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde4aa5b700 1 -- 172.21.15.72:0/181955203 shutdown_connections 2023-10-18T18:02:42.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde4aa5b700 1 --2- 172.21.15.72:0/181955203 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde440bdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:42.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde4aa5b700 1 --2- 172.21.15.72:0/181955203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde44067ff0 0x7fde440683d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:42.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde4aa5b700 1 -- 172.21.15.72:0/181955203 >> 172.21.15.72:0/181955203 conn(0x7fde440620d0 msgr2=0x7fde440624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:42.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde4aa5b700 1 -- 172.21.15.72:0/181955203 shutdown_connections 2023-10-18T18:02:42.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.316+0000 7fde4aa5b700 1 -- 172.21.15.72:0/181955203 wait complete. 2023-10-18T18:02:42.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde4aa5b700 1 Processor -- start 2023-10-18T18:02:42.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde4aa5b700 1 -- start start 2023-10-18T18:02:42.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde4aa5b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde44162790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:42.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde4aa5b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde44067ff0 0x7fde44162c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:42.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde4aa5b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde440bca50 0x7fde44145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:42.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde4aa5b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440bb910 con 0x7fde44067ff0 2023-10-18T18:02:42.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde4aa5b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440c2530 con 0x7fde440bca50 2023-10-18T18:02:42.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde4aa5b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde44163170 con 0x7fde44065e30 2023-10-18T18:02:42.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde3b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde44067ff0 0x7fde44162c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:42.335 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde43fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde44162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:42.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde3b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde44067ff0 0x7fde44162c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45118/0 (socket says 172.21.15.72:45118) 2023-10-18T18:02:42.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde43fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde44162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:35944/0 (socket says 172.21.15.72:35944) 2023-10-18T18:02:42.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde3b7fe700 1 -- 172.21.15.72:0/3237134185 learned_addr learned my addr 172.21.15.72:0/3237134185 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:42.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde48ff8700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde440bca50 0x7fde44145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:42.336 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde43fff700 1 -- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde440bca50 msgr2=0x7fde44145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:42.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde43fff700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde440bca50 0x7fde44145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:42.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde43fff700 1 -- 172.21.15.72:0/3237134185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde44067ff0 msgr2=0x7fde44162c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:42.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde43fff700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde44067ff0 0x7fde44162c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:42.337 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde43fff700 1 -- 172.21.15.72:0/3237134185 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34009370 con 0x7fde44065e30 2023-10-18T18:02:42.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde3b7fe700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde44067ff0 0x7fde44162c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:42.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.320+0000 7fde43fff700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde44162790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fde34006fd0 tx=0x7fde34017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:42.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.324+0000 7fde40ff9700 1 -- 172.21.15.72:0/3237134185 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34006d90 con 0x7fde44065e30 2023-10-18T18:02:42.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.324+0000 7fde40ff9700 1 -- 172.21.15.72:0/3237134185 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3400beb0 con 0x7fde44065e30 2023-10-18T18:02:42.338 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.324+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde440c1e60 con 0x7fde44065e30 2023-10-18T18:02:42.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.324+0000 7fde40ff9700 1 -- 172.21.15.72:0/3237134185 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3400baf0 con 0x7fde44065e30 2023-10-18T18:02:42.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.324+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde440c0db0 con 0x7fde44065e30 2023-10-18T18:02:42.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.324+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4406c190 con 0x7fde44065e30 2023-10-18T18:02:42.339 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.324+0000 7fde40ff9700 1 -- 172.21.15.72:0/3237134185 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fde3403bb90 con 0x7fde44065e30 2023-10-18T18:02:42.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.328+0000 7fde40ff9700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fde2405a540 0x7fde2405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:42.340 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.328+0000 7fde40ff9700 1 -- 172.21.15.72:0/3237134185 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fde34075440 con 0x7fde44065e30 2023-10-18T18:02:42.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.328+0000 7fde3b7fe700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fde2405a540 0x7fde2405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:42.341 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.328+0000 7fde3b7fe700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fde2405a540 0x7fde2405ca00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fde440c20b0 tx=0x7fde2c00b040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:42.342 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.336+0000 7fde40ff9700 1 -- 172.21.15.72:0/3237134185 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fde34048ac0 con 0x7fde44065e30 2023-10-18T18:02:42.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.532+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fde44062930 con 0x7fde44065e30 2023-10-18T18:02:42.541 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde40ff9700 1 -- 172.21.15.72:0/3237134185 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fde34021080 con 0x7fde44065e30 2023-10-18T18:02:42.542 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:42.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fde2405a540 msgr2=0x7fde2405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:42.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fde2405a540 0x7fde2405ca00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fde440c20b0 tx=0x7fde2c00b040).stop 2023-10-18T18:02:42.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 msgr2=0x7fde44162790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:42.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde44162790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fde34006fd0 tx=0x7fde34017000).stop 2023-10-18T18:02:42.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 shutdown_connections 2023-10-18T18:02:42.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fde2405a540 0x7fde2405ca00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:42.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde440bca50 0x7fde44145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:42.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde44067ff0 0x7fde44162c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:42.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 --2- 172.21.15.72:0/3237134185 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fde44065e30 0x7fde44162790 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:42.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.536+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 >> 172.21.15.72:0/3237134185 conn(0x7fde440620d0 msgr2=0x7fde4414cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:42.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.540+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 shutdown_connections 2023-10-18T18:02:42.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:42.540+0000 7fde4aa5b700 1 -- 172.21.15.72:0/3237134185 wait complete. 2023-10-18T18:02:42.563 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:42.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~251s 2023-10-18T18:02:42.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~197s 2023-10-18T18:02:42.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~151s 2023-10-18T18:02:42.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~177s 2023-10-18T18:02:42.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~164s 2023-10-18T18:02:42.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~211s 2023-10-18T18:02:45.565 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:45.721 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.716+0000 7f180d347700 1 Processor -- start 2023-10-18T18:02:45.722 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.716+0000 7f180d347700 1 -- start start 2023-10-18T18:02:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.716+0000 7f180d347700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f18081517c0 0x7f180806a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18080630e0 0x7f1808065700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1808154b00 con 0x7f18081513e0 2023-10-18T18:02:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1808069380 con 0x7f18081517c0 2023-10-18T18:02:45.725 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1808154f90 con 0x7f18080630e0 2023-10-18T18:02:45.726 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f17ff7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f18081517c0 0x7f180806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f1807fff700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f18081513e0 0x7f1808067d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:37812/0 (socket says 172.21.15.72:37812) 2023-10-18T18:02:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f1807fff700 1 -- 172.21.15.72:0/1528183280 learned_addr learned my addr 172.21.15.72:0/1528183280 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180cb46700 1 --2- 172.21.15.72:0/1528183280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18080630e0 0x7f1808065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f17ff7fe700 1 -- 172.21.15.72:0/1528183280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18080630e0 msgr2=0x7f1808065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:45.727 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f17ff7fe700 1 --2- 172.21.15.72:0/1528183280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18080630e0 0x7f1808065700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:45.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f17ff7fe700 1 -- 172.21.15.72:0/1528183280 >> v1:172.21.15.72:6789/0 conn(0x7f18081513e0 legacy=0x7f1808067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:45.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f17ff7fe700 1 -- 172.21.15.72:0/1528183280 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1808068570 con 0x7f18081517c0 2023-10-18T18:02:45.728 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f17ff7fe700 1 --2- 172.21.15.72:0/1528183280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18081517c0 0x7f180806a040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f17f0009760 tx=0x7f17f0009060).ready entity=mon.1 client_cookie=b5c9ade7c04cc4b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:45.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f18077fe700 1 -- 172.21.15.72:0/1528183280 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f000f510 con 0x7f18081517c0 2023-10-18T18:02:45.729 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f18077fe700 1 -- 172.21.15.72:0/1528183280 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f000f670 con 0x7f18081517c0 2023-10-18T18:02:45.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f18077fe700 1 -- 172.21.15.72:0/1528183280 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f000f9f0 con 0x7f18081517c0 2023-10-18T18:02:45.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 -- 172.21.15.72:0/1528183280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18081517c0 msgr2=0x7f180806a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:45.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 --2- 172.21.15.72:0/1528183280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18081517c0 0x7f180806a040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f17f0009760 tx=0x7f17f0009060).stop 2023-10-18T18:02:45.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 -- 172.21.15.72:0/1528183280 shutdown_connections 2023-10-18T18:02:45.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 --2- 172.21.15.72:0/1528183280 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18080630e0 0x7f1808065700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:45.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 --2- 172.21.15.72:0/1528183280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18081517c0 0x7f180806a040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:45.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 -- 172.21.15.72:0/1528183280 >> 172.21.15.72:0/1528183280 conn(0x7f1808062ba0 msgr2=0x7f18080c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:45.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 -- 172.21.15.72:0/1528183280 shutdown_connections 2023-10-18T18:02:45.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.720+0000 7f180d347700 1 -- 172.21.15.72:0/1528183280 wait complete. 2023-10-18T18:02:45.732 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f180d347700 1 Processor -- start 2023-10-18T18:02:45.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f180d347700 1 -- start start 2023-10-18T18:02:45.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f180d347700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18080630e0 0x7f1808157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:45.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f180d347700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18081513e0 0x7f180814c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:45.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f180d347700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18081517c0 0x7f18081562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:45.733 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f180d347700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1808156150 con 0x7f18080630e0 2023-10-18T18:02:45.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f180d347700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180815c4b0 con 0x7f18081517c0 2023-10-18T18:02:45.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f180d347700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1808164910 con 0x7f18081513e0 2023-10-18T18:02:45.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f1807fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18080630e0 0x7f1808157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:45.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f1807fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18080630e0 0x7f1808157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45132/0 (socket says 172.21.15.72:45132) 2023-10-18T18:02:45.734 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f1807fff700 1 -- 172.21.15.72:0/2938241729 learned_addr learned my addr 172.21.15.72:0/2938241729 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:45.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f17ff7fe700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18081513e0 0x7f180814c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:45.735 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.724+0000 7f180cb46700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18081517c0 0x7f18081562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:45.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1807fff700 1 -- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18081513e0 msgr2=0x7f180814c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:45.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1807fff700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18081513e0 0x7f180814c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:45.736 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1807fff700 1 -- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18081517c0 msgr2=0x7f18081562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:45.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1807fff700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18081517c0 0x7f18081562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:45.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1807fff700 1 -- 172.21.15.72:0/2938241729 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f00092a0 con 0x7f18080630e0 2023-10-18T18:02:45.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1807fff700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18080630e0 0x7f1808157c60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f17f8007fd0 tx=0x7f17f8007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:45.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1804ff9700 1 -- 172.21.15.72:0/2938241729 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f800ce10 con 0x7f18080630e0 2023-10-18T18:02:45.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1804ff9700 1 -- 172.21.15.72:0/2938241729 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f80072e0 con 0x7f18080630e0 2023-10-18T18:02:45.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1804ff9700 1 -- 172.21.15.72:0/2938241729 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f8007660 con 0x7f18080630e0 2023-10-18T18:02:45.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1808156e00 con 0x7f18080630e0 2023-10-18T18:02:45.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1808164b20 con 0x7f18080630e0 2023-10-18T18:02:45.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1808046f00 con 0x7f18080630e0 2023-10-18T18:02:45.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.728+0000 7f1804ff9700 1 -- 172.21.15.72:0/2938241729 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f17f80059b0 con 0x7f18080630e0 2023-10-18T18:02:45.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.732+0000 7f1804ff9700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f17e805a510 0x7f17e805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:45.739 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.732+0000 7f1804ff9700 1 -- 172.21.15.72:0/2938241729 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f17f8074090 con 0x7f18080630e0 2023-10-18T18:02:45.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.732+0000 7f17ff7fe700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f17e805a510 0x7f17e805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:45.740 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.732+0000 7f17ff7fe700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f17e805a510 0x7f17e805c9d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f17f0009fd0 tx=0x7f17f0017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:45.745 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.740+0000 7f1804ff9700 1 -- 172.21.15.72:0/2938241729 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f17f80475e0 con 0x7f18080630e0 2023-10-18T18:02:45.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.932+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f18081541d0 con 0x7f18080630e0 2023-10-18T18:02:45.938 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.932+0000 7f1804ff9700 1 -- 172.21.15.72:0/2938241729 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f17f804ac00 con 0x7f18080630e0 2023-10-18T18:02:45.939 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:45.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f17e805a510 msgr2=0x7f17e805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:45.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f17e805a510 0x7f17e805c9d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f17f0009fd0 tx=0x7f17f0017000).stop 2023-10-18T18:02:45.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18080630e0 msgr2=0x7f1808157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:45.943 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18080630e0 0x7f1808157c60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f17f8007fd0 tx=0x7f17f8007df0).stop 2023-10-18T18:02:45.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 shutdown_connections 2023-10-18T18:02:45.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f17e805a510 0x7f17e805c9d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:45.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18081517c0 0x7f18081562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:45.944 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f18081513e0 0x7f180814c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:45.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 --2- 172.21.15.72:0/2938241729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f18080630e0 0x7f1808157c60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:45.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 >> 172.21.15.72:0/2938241729 conn(0x7f1808062ba0 msgr2=0x7f18080bccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:45.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 shutdown_connections 2023-10-18T18:02:45.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:45.936+0000 7f180d347700 1 -- 172.21.15.72:0/2938241729 wait complete. 2023-10-18T18:02:45.964 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:48.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~258s 2023-10-18T18:02:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~203s 2023-10-18T18:02:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~158s 2023-10-18T18:02:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~184s 2023-10-18T18:02:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~171s 2023-10-18T18:02:48.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~217s 2023-10-18T18:02:48.965 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:49.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.120+0000 7fa455f79700 1 Processor -- start 2023-10-18T18:02:49.129 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.124+0000 7fa455f79700 1 -- start start 2023-10-18T18:02:49.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.124+0000 7fa455f79700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa45014d050 0x7fa450066c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:49.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.124+0000 7fa455f79700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa450150620 con 0x7fa450063200 2023-10-18T18:02:49.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.124+0000 7fa455f79700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa450150900 con 0x7fa450062dc0 2023-10-18T18:02:49.132 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.124+0000 7fa455f79700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa450066890 con 0x7fa45014d050 2023-10-18T18:02:49.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.124+0000 7fa455778700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fa450063200 0x7fa4500bd120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:37828/0 (socket says 172.21.15.72:37828) 2023-10-18T18:02:49.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.124+0000 7fa455778700 1 -- 172.21.15.72:0/3210811134 learned_addr learned my addr 172.21.15.72:0/3210811134 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:49.133 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.124+0000 7fa454f77700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa45014d050 0x7fa450066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:49.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa4477fe700 1 -- 172.21.15.72:0/3210811134 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080601804 0 0) 0x7fa450150620 con 0x7fa450063200 2023-10-18T18:02:49.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa4477fe700 1 -- 172.21.15.72:0/3210811134 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4380033f0 con 0x7fa450063200 2023-10-18T18:02:49.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa4477fe700 1 -- 172.21.15.72:0/3210811134 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970025224 0 0) 0x7fa450150900 con 0x7fa450062dc0 2023-10-18T18:02:49.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa4477fe700 1 -- 172.21.15.72:0/3210811134 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa450150620 con 0x7fa450062dc0 2023-10-18T18:02:49.135 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa454f77700 1 -- 172.21.15.72:0/3210811134 >> v1:172.21.15.179:6789/0 conn(0x7fa450062dc0 legacy=0x7fa45014fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa454f77700 1 -- 172.21.15.72:0/3210811134 >> v1:172.21.15.72:6789/0 conn(0x7fa450063200 legacy=0x7fa4500bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa454f77700 1 -- 172.21.15.72:0/3210811134 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500622e0 con 0x7fa45014d050 2023-10-18T18:02:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa454f77700 1 --2- 172.21.15.72:0/3210811134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa45014d050 0x7fa450066c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fa44c009790 tx=0x7fa44c009060).ready entity=mon.2 client_cookie=981912134b819bb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa4477fe700 1 -- 172.21.15.72:0/3210811134 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00f510 con 0x7fa45014d050 2023-10-18T18:02:49.136 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa4477fe700 1 -- 172.21.15.72:0/3210811134 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c00f670 con 0x7fa45014d050 2023-10-18T18:02:49.137 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa4477fe700 1 -- 172.21.15.72:0/3210811134 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00f9f0 con 0x7fa45014d050 2023-10-18T18:02:49.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa455f79700 1 -- 172.21.15.72:0/3210811134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa45014d050 msgr2=0x7fa450066c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:49.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa455f79700 1 --2- 172.21.15.72:0/3210811134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa45014d050 0x7fa450066c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fa44c009790 tx=0x7fa44c009060).stop 2023-10-18T18:02:49.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa455f79700 1 -- 172.21.15.72:0/3210811134 shutdown_connections 2023-10-18T18:02:49.138 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa455f79700 1 --2- 172.21.15.72:0/3210811134 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa45014d050 0x7fa450066c40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:49.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa455f79700 1 -- 172.21.15.72:0/3210811134 >> 172.21.15.72:0/3210811134 conn(0x7fa4500c2780 msgr2=0x7fa4500c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:49.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa455f79700 1 -- 172.21.15.72:0/3210811134 shutdown_connections 2023-10-18T18:02:49.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.128+0000 7fa455f79700 1 -- 172.21.15.72:0/3210811134 wait complete. 2023-10-18T18:02:49.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455f79700 1 Processor -- start 2023-10-18T18:02:49.139 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455f79700 1 -- start start 2023-10-18T18:02:49.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455f79700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa450062dc0 0x7fa4500b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:49.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455f79700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa450063200 0x7fa4501635e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:49.141 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455f79700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa45014d050 0x7fa4500b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:49.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455f79700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45011c270 con 0x7fa45014d050 2023-10-18T18:02:49.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455f79700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500b86d0 con 0x7fa450062dc0 2023-10-18T18:02:49.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455f79700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500b9050 con 0x7fa450063200 2023-10-18T18:02:49.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455778700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa45014d050 0x7fa4500b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:49.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa454f77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa450062dc0 0x7fa4500b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:49.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa447fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa450063200 0x7fa4501635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:49.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455778700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa45014d050 0x7fa4500b7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45148/0 (socket says 172.21.15.72:45148) 2023-10-18T18:02:49.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa454f77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa450062dc0 0x7fa4500b8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:48830/0 (socket says 172.21.15.72:48830) 2023-10-18T18:02:49.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455778700 1 -- 172.21.15.72:0/4085070874 learned_addr learned my addr 172.21.15.72:0/4085070874 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:49.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa454f77700 1 -- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa450063200 msgr2=0x7fa4501635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:49.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa454f77700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa450063200 0x7fa4501635e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:49.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa454f77700 1 -- 172.21.15.72:0/4085070874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa45014d050 msgr2=0x7fa4500b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:49.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa454f77700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa45014d050 0x7fa4500b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:49.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa454f77700 1 -- 172.21.15.72:0/4085070874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0092a0 con 0x7fa450062dc0 2023-10-18T18:02:49.144 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa455778700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa45014d050 0x7fa4500b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:02:49.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa454f77700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa450062dc0 0x7fa4500b8b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fa450063a90 tx=0x7fa44c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:49.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa444ff9700 1 -- 172.21.15.72:0/4085070874 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00fe80 con 0x7fa450062dc0 2023-10-18T18:02:49.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa444ff9700 1 -- 172.21.15.72:0/4085070874 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c00f7e0 con 0x7fa450062dc0 2023-10-18T18:02:49.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.132+0000 7fa444ff9700 1 -- 172.21.15.72:0/4085070874 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c01b640 con 0x7fa450062dc0 2023-10-18T18:02:49.145 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.136+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa450163db0 con 0x7fa450062dc0 2023-10-18T18:02:49.146 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.136+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa450164370 con 0x7fa450062dc0 2023-10-18T18:02:49.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.136+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa450046f00 con 0x7fa450062dc0 2023-10-18T18:02:49.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.136+0000 7fa444ff9700 1 -- 172.21.15.72:0/4085070874 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fa44c01b9b0 con 0x7fa450062dc0 2023-10-18T18:02:49.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.136+0000 7fa444ff9700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa43c05a510 0x7fa43c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:49.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.140+0000 7fa444ff9700 1 -- 172.21.15.72:0/4085070874 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fa44c075f20 con 0x7fa450062dc0 2023-10-18T18:02:49.147 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.140+0000 7fa447fff700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa43c05a510 0x7fa43c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:49.148 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.140+0000 7fa447fff700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa43c05a510 0x7fa43c05c9d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fa4500b97a0 tx=0x7fa440007bd0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:49.157 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.152+0000 7fa444ff9700 1 -- 172.21.15.72:0/4085070874 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fa44c049570 con 0x7fa450062dc0 2023-10-18T18:02:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.344+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa450063730 con 0x7fa450062dc0 2023-10-18T18:02:49.350 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.344+0000 7fa444ff9700 1 -- 172.21.15.72:0/4085070874 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fa44c04cb50 con 0x7fa450062dc0 2023-10-18T18:02:49.351 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:49.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa43c05a510 msgr2=0x7fa43c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:49.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa43c05a510 0x7fa43c05c9d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fa4500b97a0 tx=0x7fa440007bd0).stop 2023-10-18T18:02:49.355 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa450062dc0 msgr2=0x7fa4500b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa450062dc0 0x7fa4500b8b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fa450063a90 tx=0x7fa44c017000).stop 2023-10-18T18:02:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 shutdown_connections 2023-10-18T18:02:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fa43c05a510 0x7fa43c05c9d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa45014d050 0x7fa4500b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:49.356 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fa450063200 0x7fa4501635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:49.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 --2- 172.21.15.72:0/4085070874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa450062dc0 0x7fa4500b8b60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:49.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 >> 172.21.15.72:0/4085070874 conn(0x7fa4500c2780 msgr2=0x7fa45014c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:49.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 shutdown_connections 2023-10-18T18:02:49.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:49.348+0000 7fa455f79700 1 -- 172.21.15.72:0/4085070874 wait complete. 2023-10-18T18:02:49.378 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:52.379 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:52.542 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea950b9700 1 Processor -- start 2023-10-18T18:02:52.543 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea950b9700 1 -- start start 2023-10-18T18:02:52.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea950b9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fea901513e0 0x7fea90067d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:52.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea950b9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901517c0 0x7fea9006a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:52.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea950b9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea90154b00 con 0x7fea900630e0 2023-10-18T18:02:52.544 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea950b9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea90069380 con 0x7fea901513e0 2023-10-18T18:02:52.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea950b9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea90154f90 con 0x7fea901517c0 2023-10-18T18:02:52.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea87fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901517c0 0x7fea9006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:52.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea87fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901517c0 0x7fea9006a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:47810/0 (socket says 172.21.15.72:47810) 2023-10-18T18:02:52.546 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.536+0000 7fea87fff700 1 -- 172.21.15.72:0/3440257214 learned_addr learned my addr 172.21.15.72:0/3440257214 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:52.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea8ffff700 1 --2- 172.21.15.72:0/3440257214 >> v2:172.21.15.72:3300/0 conn(0x7fea901513e0 0x7fea90067d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:52.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea87fff700 1 -- 172.21.15.72:0/3440257214 >> v2:172.21.15.72:3300/0 conn(0x7fea901513e0 msgr2=0x7fea90067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:52.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea87fff700 1 --2- 172.21.15.72:0/3440257214 >> v2:172.21.15.72:3300/0 conn(0x7fea901513e0 0x7fea90067d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:52.547 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea87fff700 1 -- 172.21.15.72:0/3440257214 >> v1:172.21.15.72:6789/0 conn(0x7fea900630e0 legacy=0x7fea90065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:52.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea87fff700 1 -- 172.21.15.72:0/3440257214 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea90068570 con 0x7fea901517c0 2023-10-18T18:02:52.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea8ffff700 1 --2- 172.21.15.72:0/3440257214 >> v2:172.21.15.72:3300/0 conn(0x7fea901513e0 0x7fea90067d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-10-18T18:02:52.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea87fff700 1 --2- 172.21.15.72:0/3440257214 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901517c0 0x7fea9006a040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fea78006fd0 tx=0x7fea780090f0).ready entity=mon.2 client_cookie=c0224790148ab532 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:52.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea8f7fe700 1 -- 172.21.15.72:0/3440257214 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800b690 con 0x7fea901517c0 2023-10-18T18:02:52.548 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea8f7fe700 1 -- 172.21.15.72:0/3440257214 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800b7f0 con 0x7fea901517c0 2023-10-18T18:02:52.549 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea8f7fe700 1 -- 172.21.15.72:0/3440257214 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800bb70 con 0x7fea901517c0 2023-10-18T18:02:52.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea950b9700 1 -- 172.21.15.72:0/3440257214 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901517c0 msgr2=0x7fea9006a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:52.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea950b9700 1 --2- 172.21.15.72:0/3440257214 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901517c0 0x7fea9006a040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fea78006fd0 tx=0x7fea780090f0).stop 2023-10-18T18:02:52.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea950b9700 1 -- 172.21.15.72:0/3440257214 shutdown_connections 2023-10-18T18:02:52.550 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea950b9700 1 --2- 172.21.15.72:0/3440257214 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901517c0 0x7fea9006a040 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:52.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea950b9700 1 --2- 172.21.15.72:0/3440257214 >> v2:172.21.15.72:3300/0 conn(0x7fea901513e0 0x7fea90067d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:52.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea950b9700 1 -- 172.21.15.72:0/3440257214 >> 172.21.15.72:0/3440257214 conn(0x7fea90062ba0 msgr2=0x7fea900c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:52.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea950b9700 1 -- 172.21.15.72:0/3440257214 shutdown_connections 2023-10-18T18:02:52.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.540+0000 7fea950b9700 1 -- 172.21.15.72:0/3440257214 wait complete. 2023-10-18T18:02:52.551 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea950b9700 1 Processor -- start 2023-10-18T18:02:52.552 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea950b9700 1 -- start start 2023-10-18T18:02:52.553 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea950b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea900630e0 0x7fea901681a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:52.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea950b9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901513e0 0x7fea9014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:52.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea950b9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea901517c0 0x7fea9015b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:52.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea950b9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90157a70 con 0x7fea900630e0 2023-10-18T18:02:52.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea950b9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90167d10 con 0x7fea901517c0 2023-10-18T18:02:52.554 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea950b9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900bb5f0 con 0x7fea901513e0 2023-10-18T18:02:52.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea8ffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea900630e0 0x7fea901681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:52.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea87fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901513e0 0x7fea9014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:52.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea948b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea901517c0 0x7fea9015b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:52.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea87fff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901513e0 0x7fea9014edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:47814/0 (socket says 172.21.15.72:47814) 2023-10-18T18:02:52.555 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea8ffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea900630e0 0x7fea901681a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49566/0 (socket says 172.21.15.72:49566) 2023-10-18T18:02:52.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea87fff700 1 -- 172.21.15.72:0/2805899624 learned_addr learned my addr 172.21.15.72:0/2805899624 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:52.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea8ffff700 1 -- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901513e0 msgr2=0x7fea9014edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:52.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea8ffff700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901513e0 0x7fea9014edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:52.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea8ffff700 1 -- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea901517c0 msgr2=0x7fea9015b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:52.556 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea8ffff700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea901517c0 0x7fea9015b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:52.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea8ffff700 1 -- 172.21.15.72:0/2805899624 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea78009370 con 0x7fea900630e0 2023-10-18T18:02:52.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.544+0000 7fea8ffff700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea900630e0 0x7fea901681a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fea8000c880 tx=0x7fea80008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:52.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.548+0000 7fea8cff9700 1 -- 172.21.15.72:0/2805899624 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80010580 con 0x7fea900630e0 2023-10-18T18:02:52.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.548+0000 7fea8cff9700 1 -- 172.21.15.72:0/2805899624 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea800106e0 con 0x7fea900630e0 2023-10-18T18:02:52.557 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.548+0000 7fea8cff9700 1 -- 172.21.15.72:0/2805899624 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80010a60 con 0x7fea900630e0 2023-10-18T18:02:52.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.548+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea900bb8d0 con 0x7fea900630e0 2023-10-18T18:02:52.558 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.548+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea900bbe90 con 0x7fea900630e0 2023-10-18T18:02:52.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.548+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea90046f00 con 0x7fea900630e0 2023-10-18T18:02:52.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.548+0000 7fea8cff9700 1 -- 172.21.15.72:0/2805899624 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fea8001b460 con 0x7fea900630e0 2023-10-18T18:02:52.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.552+0000 7fea8cff9700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fea7005a510 0x7fea7005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:52.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.552+0000 7fea8cff9700 1 -- 172.21.15.72:0/2805899624 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fea80074860 con 0x7fea900630e0 2023-10-18T18:02:52.559 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.552+0000 7fea87fff700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fea7005a510 0x7fea7005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:52.560 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.552+0000 7fea87fff700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fea7005a510 0x7fea7005c9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fea78006ec0 tx=0x7fea78017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:52.563 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.556+0000 7fea8cff9700 1 -- 172.21.15.72:0/2805899624 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fea80047e10 con 0x7fea900630e0 2023-10-18T18:02:52.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.756+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fea900661a0 con 0x7fea900630e0 2023-10-18T18:02:52.763 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.756+0000 7fea8cff9700 1 -- 172.21.15.72:0/2805899624 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fea8004b430 con 0x7fea900630e0 2023-10-18T18:02:52.764 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:52.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fea7005a510 msgr2=0x7fea7005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:52.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fea7005a510 0x7fea7005c9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fea78006ec0 tx=0x7fea78017000).stop 2023-10-18T18:02:52.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea900630e0 msgr2=0x7fea901681a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:52.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea900630e0 0x7fea901681a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fea8000c880 tx=0x7fea80008040).stop 2023-10-18T18:02:52.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 shutdown_connections 2023-10-18T18:02:52.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fea7005a510 0x7fea7005c9d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:52.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea901517c0 0x7fea9015b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:52.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fea901513e0 0x7fea9014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:52.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 --2- 172.21.15.72:0/2805899624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea900630e0 0x7fea901681a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:52.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 >> 172.21.15.72:0/2805899624 conn(0x7fea90062ba0 msgr2=0x7fea90152850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:52.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 shutdown_connections 2023-10-18T18:02:52.769 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:52.760+0000 7fea950b9700 1 -- 172.21.15.72:0/2805899624 wait complete. 2023-10-18T18:02:52.786 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:55.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~264s 2023-10-18T18:02:55.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~210s 2023-10-18T18:02:55.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~164s 2023-10-18T18:02:55.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~190s 2023-10-18T18:02:55.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~177s 2023-10-18T18:02:55.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~223s 2023-10-18T18:02:55.788 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:55.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.940+0000 7f3600a1a700 1 Processor -- start 2023-10-18T18:02:55.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.940+0000 7f3600a1a700 1 -- start start 2023-10-18T18:02:55.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.940+0000 7f3600a1a700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f35fc1517c0 0x7f35fc06a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:55.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.940+0000 7f3600a1a700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc0630e0 0x7f35fc065700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:55.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.940+0000 7f3600a1a700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc154b00 con 0x7f35fc1513e0 2023-10-18T18:02:55.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.940+0000 7f3600a1a700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc069380 con 0x7f35fc1517c0 2023-10-18T18:02:55.949 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.940+0000 7f3600a1a700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc154f90 con 0x7f35fc0630e0 2023-10-18T18:02:55.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35fad9d700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f35fc1513e0 0x7f35fc067d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36882/0 (socket says 172.21.15.72:36882) 2023-10-18T18:02:55.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35fad9d700 1 -- 172.21.15.72:0/1128796545 learned_addr learned my addr 172.21.15.72:0/1128796545 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:55.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35f259c700 1 --2- 172.21.15.72:0/1128796545 >> v2:172.21.15.179:3300/0 conn(0x7f35fc1517c0 0x7f35fc06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:55.950 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35fb59e700 1 --2- 172.21.15.72:0/1128796545 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc0630e0 0x7f35fc065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:55.951 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35f259c700 1 -- 172.21.15.72:0/1128796545 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc0630e0 msgr2=0x7f35fc065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:55.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35f259c700 1 --2- 172.21.15.72:0/1128796545 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc0630e0 0x7f35fc065700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:55.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35f259c700 1 -- 172.21.15.72:0/1128796545 >> v1:172.21.15.72:6789/0 conn(0x7f35fc1513e0 legacy=0x7f35fc067d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:55.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35fa59c700 1 -- 172.21.15.72:0/1128796545 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515540405 0 0) 0x7f35fc154b00 con 0x7f35fc1513e0 2023-10-18T18:02:55.952 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35f259c700 1 -- 172.21.15.72:0/1128796545 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc068570 con 0x7f35fc1517c0 2023-10-18T18:02:55.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35f259c700 1 --2- 172.21.15.72:0/1128796545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35fc1517c0 0x7f35fc06a040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f35e4009760 tx=0x7f35e4009060).ready entity=mon.1 client_cookie=c1a71c7bfde506f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:55.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35fa59c700 1 -- 172.21.15.72:0/1128796545 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400f510 con 0x7f35fc1517c0 2023-10-18T18:02:55.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35fa59c700 1 -- 172.21.15.72:0/1128796545 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e400f670 con 0x7f35fc1517c0 2023-10-18T18:02:55.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f35fa59c700 1 -- 172.21.15.72:0/1128796545 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400f9f0 con 0x7f35fc1517c0 2023-10-18T18:02:55.953 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f3600a1a700 1 -- 172.21.15.72:0/1128796545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35fc1517c0 msgr2=0x7f35fc06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:55.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f3600a1a700 1 --2- 172.21.15.72:0/1128796545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35fc1517c0 0x7f35fc06a040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f35e4009760 tx=0x7f35e4009060).stop 2023-10-18T18:02:55.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f3600a1a700 1 -- 172.21.15.72:0/1128796545 shutdown_connections 2023-10-18T18:02:55.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f3600a1a700 1 --2- 172.21.15.72:0/1128796545 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc0630e0 0x7f35fc065700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:55.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f3600a1a700 1 --2- 172.21.15.72:0/1128796545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35fc1517c0 0x7f35fc06a040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:55.954 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f3600a1a700 1 -- 172.21.15.72:0/1128796545 >> 172.21.15.72:0/1128796545 conn(0x7f35fc062ba0 msgr2=0x7f35fc0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:55.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f3600a1a700 1 -- 172.21.15.72:0/1128796545 shutdown_connections 2023-10-18T18:02:55.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.944+0000 7f3600a1a700 1 -- 172.21.15.72:0/1128796545 wait complete. 2023-10-18T18:02:55.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f3600a1a700 1 Processor -- start 2023-10-18T18:02:55.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f3600a1a700 1 -- start start 2023-10-18T18:02:55.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f3600a1a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35fc0630e0 0x7f35fc14e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:55.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f3600a1a700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc1513e0 0x7f35fc0c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:55.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f3600a1a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1517c0 0x7f35fc155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:55.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f3600a1a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1597e0 con 0x7f35fc1517c0 2023-10-18T18:02:55.957 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f3600a1a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc1558d0 con 0x7f35fc0630e0 2023-10-18T18:02:55.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f3600a1a700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc152d10 con 0x7f35fc1513e0 2023-10-18T18:02:55.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35fb59e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1517c0 0x7f35fc155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:55.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35fb59e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1517c0 0x7f35fc155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49576/0 (socket says 172.21.15.72:49576) 2023-10-18T18:02:55.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35fb59e700 1 -- 172.21.15.72:0/2838457991 learned_addr learned my addr 172.21.15.72:0/2838457991 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:55.958 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35fad9d700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35fc0630e0 0x7f35fc14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:55.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35f259c700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc1513e0 0x7f35fc0c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:55.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35fb59e700 1 -- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc1513e0 msgr2=0x7f35fc0c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:55.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35fb59e700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc1513e0 0x7f35fc0c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:55.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35fb59e700 1 -- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35fc0630e0 msgr2=0x7f35fc14e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:55.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35fb59e700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35fc0630e0 0x7f35fc14e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:55.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.948+0000 7f35fb59e700 1 -- 172.21.15.72:0/2838457991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40092a0 con 0x7f35fc1517c0 2023-10-18T18:02:55.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.952+0000 7f35fb59e700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1517c0 0x7f35fc155d60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f35f4009df0 tx=0x7f35f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:55.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.952+0000 7f35f37fe700 1 -- 172.21.15.72:0/2838457991 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400caa0 con 0x7f35fc1517c0 2023-10-18T18:02:55.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.952+0000 7f35f37fe700 1 -- 172.21.15.72:0/2838457991 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f400f040 con 0x7f35fc1517c0 2023-10-18T18:02:55.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.952+0000 7f35f37fe700 1 -- 172.21.15.72:0/2838457991 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f400ce10 con 0x7f35fc1517c0 2023-10-18T18:02:55.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.952+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc14ea10 con 0x7f35fc1517c0 2023-10-18T18:02:55.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.952+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc14efd0 con 0x7f35fc1517c0 2023-10-18T18:02:55.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.952+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc155a60 con 0x7f35fc1517c0 2023-10-18T18:02:55.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.952+0000 7f35f37fe700 1 -- 172.21.15.72:0/2838457991 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f35f4013610 con 0x7f35fc1517c0 2023-10-18T18:02:55.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.956+0000 7f35f37fe700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f35dc05a510 0x7f35dc05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:55.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.956+0000 7f35f37fe700 1 -- 172.21.15.72:0/2838457991 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f35f4075180 con 0x7f35fc1517c0 2023-10-18T18:02:55.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.956+0000 7f35fad9d700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f35dc05a510 0x7f35dc05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:55.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.956+0000 7f35fad9d700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f35dc05a510 0x7f35dc05c9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f35ec0011e0 tx=0x7f35ec007bb0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:55.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:55.960+0000 7f35f37fe700 1 -- 172.21.15.72:0/2838457991 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f35f40486d0 con 0x7f35fc1517c0 2023-10-18T18:02:56.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.156+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f35fc1541d0 con 0x7f35fc1517c0 2023-10-18T18:02:56.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.156+0000 7f35f37fe700 1 -- 172.21.15.72:0/2838457991 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f35f404bcf0 con 0x7f35fc1517c0 2023-10-18T18:02:56.162 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:56.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f35dc05a510 msgr2=0x7f35dc05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:56.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f35dc05a510 0x7f35dc05c9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f35ec0011e0 tx=0x7f35ec007bb0).stop 2023-10-18T18:02:56.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1517c0 msgr2=0x7f35fc155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:56.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1517c0 0x7f35fc155d60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f35f4009df0 tx=0x7f35f400b040).stop 2023-10-18T18:02:56.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 shutdown_connections 2023-10-18T18:02:56.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f35dc05a510 0x7f35dc05c9d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:56.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35fc1517c0 0x7f35fc155d60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:56.168 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f35fc1513e0 0x7f35fc0c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:56.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 --2- 172.21.15.72:0/2838457991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35fc0630e0 0x7f35fc14e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:56.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 >> 172.21.15.72:0/2838457991 conn(0x7f35fc062ba0 msgr2=0x7f35fc151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:56.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 shutdown_connections 2023-10-18T18:02:56.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:56.160+0000 7f3600a1a700 1 -- 172.21.15.72:0/2838457991 wait complete. 2023-10-18T18:02:56.189 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:02:56.988 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:02:56.996 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:02:59.190 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:02:59.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.348+0000 7f661025b700 1 Processor -- start 2023-10-18T18:02:59.353 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.348+0000 7f661025b700 1 -- start start 2023-10-18T18:02:59.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.348+0000 7f661025b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 0x7f66080683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:59.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.348+0000 7f661025b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f66080bca50 0x7f6608150890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:59.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6608066900 con 0x7f6608067ff0 2023-10-18T18:02:59.357 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6608068a50 con 0x7f66080bca50 2023-10-18T18:02:59.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66080659a0 con 0x7f6608065e30 2023-10-18T18:02:59.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660dff7700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f6608067ff0 0x7f66080bdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:36892/0 (socket says 172.21.15.72:36892) 2023-10-18T18:02:59.358 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660dff7700 1 -- 172.21.15.72:0/19388877 learned_addr learned my addr 172.21.15.72:0/19388877 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:02:59.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660d7f6700 1 --2- 172.21.15.72:0/19388877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 0x7f66080683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:59.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660e7f8700 1 --2- 172.21.15.72:0/19388877 >> v2:172.21.15.72:3300/0 conn(0x7f66080bca50 0x7f6608150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:59.359 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660d7f6700 1 -- 172.21.15.72:0/19388877 >> v2:172.21.15.72:3300/0 conn(0x7f66080bca50 msgr2=0x7f6608150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:02:59.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660d7f6700 1 --2- 172.21.15.72:0/19388877 >> v2:172.21.15.72:3300/0 conn(0x7f66080bca50 0x7f6608150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:59.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660d7f6700 1 -- 172.21.15.72:0/19388877 >> v1:172.21.15.72:6789/0 conn(0x7f6608067ff0 legacy=0x7f66080bdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:59.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660d7f6700 1 -- 172.21.15.72:0/19388877 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6608066620 con 0x7f6608065e30 2023-10-18T18:02:59.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660cff5700 1 -- 172.21.15.72:0/19388877 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794771243 0 0) 0x7f6608066900 con 0x7f6608067ff0 2023-10-18T18:02:59.360 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660d7f6700 1 --2- 172.21.15.72:0/19388877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 0x7f66080683d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f65f8009790 tx=0x7f65f8009060).ready entity=mon.2 client_cookie=229fa3c8b2f59845 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:59.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660cff5700 1 -- 172.21.15.72:0/19388877 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800b530 con 0x7f6608065e30 2023-10-18T18:02:59.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660cff5700 1 -- 172.21.15.72:0/19388877 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f800b690 con 0x7f6608065e30 2023-10-18T18:02:59.361 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f660cff5700 1 -- 172.21.15.72:0/19388877 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800ba10 con 0x7f6608065e30 2023-10-18T18:02:59.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 -- 172.21.15.72:0/19388877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 msgr2=0x7f66080683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:59.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 --2- 172.21.15.72:0/19388877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 0x7f66080683d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f65f8009790 tx=0x7f65f8009060).stop 2023-10-18T18:02:59.362 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 -- 172.21.15.72:0/19388877 shutdown_connections 2023-10-18T18:02:59.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 --2- 172.21.15.72:0/19388877 >> v2:172.21.15.72:3300/0 conn(0x7f66080bca50 0x7f6608150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:59.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 --2- 172.21.15.72:0/19388877 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 0x7f66080683d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:59.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 -- 172.21.15.72:0/19388877 >> 172.21.15.72:0/19388877 conn(0x7f66080620d0 msgr2=0x7f66080624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:59.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 -- 172.21.15.72:0/19388877 shutdown_connections 2023-10-18T18:02:59.363 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.352+0000 7f661025b700 1 -- 172.21.15.72:0/19388877 wait complete. 2023-10-18T18:02:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.356+0000 7f661025b700 1 Processor -- start 2023-10-18T18:02:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.356+0000 7f661025b700 1 -- start start 2023-10-18T18:02:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.356+0000 7f661025b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 0x7f66080b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.356+0000 7f661025b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6608067ff0 0x7f66080b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:59.365 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.356+0000 7f661025b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66080bca50 0x7f66080c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:59.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.356+0000 7f661025b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66080b9f80 con 0x7f66080bca50 2023-10-18T18:02:59.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.356+0000 7f661025b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66080bc180 con 0x7f6608067ff0 2023-10-18T18:02:59.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.356+0000 7f661025b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66080bff50 con 0x7f6608065e30 2023-10-18T18:02:59.366 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660e7f8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66080bca50 0x7f66080c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660e7f8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66080bca50 0x7f66080c03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:49600/0 (socket says 172.21.15.72:49600) 2023-10-18T18:02:59.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660e7f8700 1 -- 172.21.15.72:0/255449487 learned_addr learned my addr 172.21.15.72:0/255449487 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:02:59.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660dff7700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 0x7f66080b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:59.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660d7f6700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6608067ff0 0x7f66080b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:59.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660e7f8700 1 -- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 msgr2=0x7f66080b95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:59.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660e7f8700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 0x7f66080b95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660e7f8700 1 -- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6608067ff0 msgr2=0x7f66080b9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660e7f8700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6608067ff0 0x7f66080b9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660e7f8700 1 -- 172.21.15.72:0/255449487 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80092a0 con 0x7f66080bca50 2023-10-18T18:02:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660d7f6700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6608067ff0 0x7f66080b9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:02:59.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f660e7f8700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66080bca50 0x7f66080c03e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f6600009f20 tx=0x7f660000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:59.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f65fdffb700 1 -- 172.21.15.72:0/255449487 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000cc80 con 0x7f66080bca50 2023-10-18T18:02:59.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f65fdffb700 1 -- 172.21.15.72:0/255449487 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660000f040 con 0x7f66080bca50 2023-10-18T18:02:59.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f65fdffb700 1 -- 172.21.15.72:0/255449487 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600013400 con 0x7f66080bca50 2023-10-18T18:02:59.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66080baa80 con 0x7f66080bca50 2023-10-18T18:02:59.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66080bb040 con 0x7f66080bca50 2023-10-18T18:02:59.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.360+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6608151fd0 con 0x7f66080bca50 2023-10-18T18:02:59.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.364+0000 7f65fdffb700 1 -- 172.21.15.72:0/255449487 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f66000137b0 con 0x7f66080bca50 2023-10-18T18:02:59.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.364+0000 7f65fdffb700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f65ec05a540 0x7f65ec05ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:02:59.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.364+0000 7f65fdffb700 1 -- 172.21.15.72:0/255449487 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f66000753e0 con 0x7f66080bca50 2023-10-18T18:02:59.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.364+0000 7f660dff7700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f65ec05a540 0x7f65ec05ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:02:59.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.364+0000 7f660dff7700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f65ec05a540 0x7f65ec05ca00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f66040011e0 tx=0x7f6604009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:02:59.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.372+0000 7f65fdffb700 1 -- 172.21.15.72:0/255449487 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f6608151fd0 con 0x7f66080bca50 2023-10-18T18:02:59.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.568+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f66080bd880 con 0x7f66080bca50 2023-10-18T18:02:59.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.568+0000 7f65fdffb700 1 -- 172.21.15.72:0/255449487 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f66080bd880 con 0x7f66080bca50 2023-10-18T18:02:59.574 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:02:59.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.568+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f65ec05a540 msgr2=0x7f65ec05ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:59.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.568+0000 7f661025b700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f65ec05a540 0x7f65ec05ca00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f66040011e0 tx=0x7f6604009040).stop 2023-10-18T18:02:59.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66080bca50 msgr2=0x7f66080c03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:02:59.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66080bca50 0x7f66080c03e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f6600009f20 tx=0x7f660000b040).stop 2023-10-18T18:02:59.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 shutdown_connections 2023-10-18T18:02:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f65ec05a540 0x7f65ec05ca00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f66080bca50 0x7f66080c03e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:59.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6608067ff0 0x7f66080b9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:59.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 --2- 172.21.15.72:0/255449487 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f6608065e30 0x7f66080b95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:02:59.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 >> 172.21.15.72:0/255449487 conn(0x7f66080620d0 msgr2=0x7f6608151320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:02:59.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 shutdown_connections 2023-10-18T18:02:59.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:02:59.572+0000 7f661025b700 1 -- 172.21.15.72:0/255449487 wait complete. 2023-10-18T18:02:59.595 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:01.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~270s 2023-10-18T18:03:01.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~216s 2023-10-18T18:03:01.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~170s 2023-10-18T18:03:01.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~196s 2023-10-18T18:03:01.539 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~183s 2023-10-18T18:03:01.539 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~230s 2023-10-18T18:03:02.596 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:02.761 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.756+0000 7f3a86c1b700 1 Processor -- start 2023-10-18T18:03:02.762 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.756+0000 7f3a86c1b700 1 -- start start 2023-10-18T18:03:02.764 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.756+0000 7f3a86c1b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3a80067ff0 0x7f3a800683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.756+0000 7f3a86c1b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a80065cf0 con 0x7f3a800666b0 2023-10-18T18:03:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.756+0000 7f3a86c1b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a80150350 con 0x7f3a8014cfa0 2023-10-18T18:03:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.756+0000 7f3a86c1b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a800650e0 con 0x7f3a80067ff0 2023-10-18T18:03:02.765 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a849b7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3a80067ff0 0x7f3a800683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a849b7700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f3a80067ff0 0x7f3a800683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54656/0 (socket says 172.21.15.72:54656) 2023-10-18T18:03:02.766 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a849b7700 1 -- 172.21.15.72:0/136669166 learned_addr learned my addr 172.21.15.72:0/136669166 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a849b7700 1 -- 172.21.15.72:0/136669166 >> v1:172.21.15.179:6789/0 conn(0x7f3a8014cfa0 legacy=0x7f3a8014d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a849b7700 1 -- 172.21.15.72:0/136669166 >> v1:172.21.15.72:6789/0 conn(0x7f3a800666b0 legacy=0x7f3a800c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:02.767 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a7f7fe700 1 -- 172.21.15.72:0/136669166 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1710563296 0 0) 0x7f3a80150350 con 0x7f3a8014cfa0 2023-10-18T18:03:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a849b7700 1 -- 172.21.15.72:0/136669166 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800653c0 con 0x7f3a80067ff0 2023-10-18T18:03:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a849b7700 1 --2- 172.21.15.72:0/136669166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a80067ff0 0x7f3a800683d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f3a74006fd0 tx=0x7f3a740090f0).ready entity=mon.0 client_cookie=1fceda650b8d741f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a7f7fe700 1 -- 172.21.15.72:0/136669166 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400b690 con 0x7f3a80067ff0 2023-10-18T18:03:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a7f7fe700 1 -- 172.21.15.72:0/136669166 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7400b7f0 con 0x7f3a80067ff0 2023-10-18T18:03:02.768 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a7f7fe700 1 -- 172.21.15.72:0/136669166 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400bb70 con 0x7f3a80067ff0 2023-10-18T18:03:02.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a86c1b700 1 -- 172.21.15.72:0/136669166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a80067ff0 msgr2=0x7f3a800683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:02.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a86c1b700 1 --2- 172.21.15.72:0/136669166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a80067ff0 0x7f3a800683d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f3a74006fd0 tx=0x7f3a740090f0).stop 2023-10-18T18:03:02.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a86c1b700 1 -- 172.21.15.72:0/136669166 shutdown_connections 2023-10-18T18:03:02.770 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a86c1b700 1 --2- 172.21.15.72:0/136669166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a80067ff0 0x7f3a800683d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:02.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a86c1b700 1 -- 172.21.15.72:0/136669166 >> 172.21.15.72:0/136669166 conn(0x7f3a800620d0 msgr2=0x7f3a800624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:02.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a86c1b700 1 -- 172.21.15.72:0/136669166 shutdown_connections 2023-10-18T18:03:02.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.760+0000 7f3a86c1b700 1 -- 172.21.15.72:0/136669166 wait complete. 2023-10-18T18:03:02.771 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a86c1b700 1 Processor -- start 2023-10-18T18:03:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a86c1b700 1 -- start start 2023-10-18T18:03:02.773 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a86c1b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a800666b0 0x7f3a800bf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:02.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a86c1b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a80067ff0 0x7f3a8014c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:02.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a86c1b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a8014cfa0 0x7f3a80162cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:02.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a86c1b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800bcdb0 con 0x7f3a8014cfa0 2023-10-18T18:03:02.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a86c1b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800beda0 con 0x7f3a80067ff0 2023-10-18T18:03:02.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a86c1b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a801646a0 con 0x7f3a800666b0 2023-10-18T18:03:02.774 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a849b7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a800666b0 0x7f3a800bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:02.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a849b7700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a800666b0 0x7f3a800bf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:48204/0 (socket says 172.21.15.72:48204) 2023-10-18T18:03:02.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a849b7700 1 -- 172.21.15.72:0/3074186987 learned_addr learned my addr 172.21.15.72:0/3074186987 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:02.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a851b8700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a8014cfa0 0x7f3a80162cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:02.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a7ffff700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a80067ff0 0x7f3a8014c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:02.775 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a851b8700 1 -- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a800666b0 msgr2=0x7f3a800bf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:02.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a851b8700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a800666b0 0x7f3a800bf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:02.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a851b8700 1 -- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a80067ff0 msgr2=0x7f3a8014c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:02.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a851b8700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a80067ff0 0x7f3a8014c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:02.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a851b8700 1 -- 172.21.15.72:0/3074186987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a74009370 con 0x7f3a8014cfa0 2023-10-18T18:03:02.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.764+0000 7f3a849b7700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a800666b0 0x7f3a800bf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:03:02.776 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.768+0000 7f3a851b8700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a8014cfa0 0x7f3a80162cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f3a7800c840 tx=0x7f3a780013f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:02.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.768+0000 7f3a7cff9700 1 -- 172.21.15.72:0/3074186987 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800e4a0 con 0x7f3a8014cfa0 2023-10-18T18:03:02.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.768+0000 7f3a7cff9700 1 -- 172.21.15.72:0/3074186987 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7800e620 con 0x7f3a8014cfa0 2023-10-18T18:03:02.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.768+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a800bfc60 con 0x7f3a8014cfa0 2023-10-18T18:03:02.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.768+0000 7f3a7cff9700 1 -- 172.21.15.72:0/3074186987 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800e9e0 con 0x7f3a8014cfa0 2023-10-18T18:03:02.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.768+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a800c0220 con 0x7f3a8014cfa0 2023-10-18T18:03:02.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.768+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a800bef30 con 0x7f3a8014cfa0 2023-10-18T18:03:02.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.768+0000 7f3a7cff9700 1 -- 172.21.15.72:0/3074186987 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f3a780123f0 con 0x7f3a8014cfa0 2023-10-18T18:03:02.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.768+0000 7f3a7cff9700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3a6405a510 0x7f3a6405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:02.779 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.772+0000 7f3a7cff9700 1 -- 172.21.15.72:0/3074186987 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f3a78074260 con 0x7f3a8014cfa0 2023-10-18T18:03:02.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.772+0000 7f3a849b7700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3a6405a510 0x7f3a6405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:02.780 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.772+0000 7f3a849b7700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3a6405a510 0x7f3a6405c9d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f3a7400b450 tx=0x7f3a74017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:02.784 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.776+0000 7f3a7cff9700 1 -- 172.21.15.72:0/3074186987 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f3a78047750 con 0x7f3a8014cfa0 2023-10-18T18:03:02.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.964+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3a8006c190 con 0x7f3a8014cfa0 2023-10-18T18:03:02.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a7cff9700 1 -- 172.21.15.72:0/3074186987 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f3a7804ad70 con 0x7f3a8014cfa0 2023-10-18T18:03:02.973 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:02.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3a6405a510 msgr2=0x7f3a6405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:02.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3a6405a510 0x7f3a6405c9d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f3a7400b450 tx=0x7f3a74017000).stop 2023-10-18T18:03:02.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a8014cfa0 msgr2=0x7f3a80162cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:02.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a8014cfa0 0x7f3a80162cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f3a7800c840 tx=0x7f3a780013f0).stop 2023-10-18T18:03:02.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 shutdown_connections 2023-10-18T18:03:02.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f3a6405a510 0x7f3a6405c9d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:02.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3a8014cfa0 0x7f3a80162cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:02.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a80067ff0 0x7f3a8014c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:02.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 --2- 172.21.15.72:0/3074186987 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f3a800666b0 0x7f3a800bf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:02.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 >> 172.21.15.72:0/3074186987 conn(0x7f3a800620d0 msgr2=0x7f3a801504e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:02.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 shutdown_connections 2023-10-18T18:03:02.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:02.968+0000 7f3a86c1b700 1 -- 172.21.15.72:0/3074186987 wait complete. 2023-10-18T18:03:02.994 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:05.995 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:06.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.160+0000 7fd3ddeb9700 1 Processor -- start 2023-10-18T18:03:06.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.160+0000 7fd3ddeb9700 1 -- start start 2023-10-18T18:03:06.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3ddeb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d8063350 0x7fd3d80637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:06.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3ddeb9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d814c380 con 0x7fd3d81498d0 2023-10-18T18:03:06.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3ddeb9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d814c640 con 0x7fd3d80bc930 2023-10-18T18:03:06.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3ddeb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d814c920 con 0x7fd3d8063350 2023-10-18T18:03:06.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3dceb7700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7fd3d81498d0 0x7fd3d80bd370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:52262/0 (socket says 172.21.15.72:52262) 2023-10-18T18:03:06.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3dceb7700 1 -- 172.21.15.72:0/432655426 learned_addr learned my addr 172.21.15.72:0/432655426 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:03:06.172 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3cffff700 1 --1- 172.21.15.72:0/432655426 >> v1:172.21.15.179:6789/0 conn(0x7fd3d80bc930 0x7fd3d80bdeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:56984/0 (socket says 172.21.15.72:56984) 2023-10-18T18:03:06.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3dd6b8700 1 --2- 172.21.15.72:0/432655426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d8063350 0x7fd3d80637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:06.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3cf7fe700 1 -- 172.21.15.72:0/432655426 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785158701 0 0) 0x7fd3d814c640 con 0x7fd3d80bc930 2023-10-18T18:03:06.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3cf7fe700 1 -- 172.21.15.72:0/432655426 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3c00033f0 con 0x7fd3d80bc930 2023-10-18T18:03:06.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3cf7fe700 1 -- 172.21.15.72:0/432655426 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554158923 0 0) 0x7fd3d814c380 con 0x7fd3d81498d0 2023-10-18T18:03:06.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3cf7fe700 1 -- 172.21.15.72:0/432655426 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d814c640 con 0x7fd3d81498d0 2023-10-18T18:03:06.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3dd6b8700 1 -- 172.21.15.72:0/432655426 >> v1:172.21.15.179:6789/0 conn(0x7fd3d80bc930 legacy=0x7fd3d80bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:06.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3dd6b8700 1 -- 172.21.15.72:0/432655426 >> v1:172.21.15.72:6789/0 conn(0x7fd3d81498d0 legacy=0x7fd3d80bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:06.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3dd6b8700 1 -- 172.21.15.72:0/432655426 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d814cc00 con 0x7fd3d8063350 2023-10-18T18:03:06.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.164+0000 7fd3dd6b8700 1 --2- 172.21.15.72:0/432655426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d8063350 0x7fd3d80637c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd3d0009790 tx=0x7fd3d0009060).ready entity=mon.2 client_cookie=dbe6669b771ead80 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:06.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3cf7fe700 1 -- 172.21.15.72:0/432655426 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000f510 con 0x7fd3d8063350 2023-10-18T18:03:06.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3cf7fe700 1 -- 172.21.15.72:0/432655426 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d000f670 con 0x7fd3d8063350 2023-10-18T18:03:06.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3cf7fe700 1 -- 172.21.15.72:0/432655426 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000f9f0 con 0x7fd3d8063350 2023-10-18T18:03:06.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/432655426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d8063350 msgr2=0x7fd3d80637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:06.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3ddeb9700 1 --2- 172.21.15.72:0/432655426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d8063350 0x7fd3d80637c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd3d0009790 tx=0x7fd3d0009060).stop 2023-10-18T18:03:06.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/432655426 shutdown_connections 2023-10-18T18:03:06.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3ddeb9700 1 --2- 172.21.15.72:0/432655426 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d8063350 0x7fd3d80637c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:06.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/432655426 >> 172.21.15.72:0/432655426 conn(0x7fd3d8062ba0 msgr2=0x7fd3d80c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:06.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/432655426 shutdown_connections 2023-10-18T18:03:06.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/432655426 wait complete. 2023-10-18T18:03:06.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.168+0000 7fd3ddeb9700 1 Processor -- start 2023-10-18T18:03:06.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ddeb9700 1 -- start start 2023-10-18T18:03:06.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ddeb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d8063350 0x7fd3d80a7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:06.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ddeb9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d80bc930 0x7fd3d814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:06.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ddeb9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d81498d0 0x7fd3d8146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:06.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ddeb9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d80a54c0 con 0x7fd3d81498d0 2023-10-18T18:03:06.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ddeb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d80a4940 con 0x7fd3d8063350 2023-10-18T18:03:06.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ddeb9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d814be40 con 0x7fd3d80bc930 2023-10-18T18:03:06.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dd6b8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d81498d0 0x7fd3d8146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:06.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dd6b8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d81498d0 0x7fd3d8146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54666/0 (socket says 172.21.15.72:54666) 2023-10-18T18:03:06.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dd6b8700 1 -- 172.21.15.72:0/109411535 learned_addr learned my addr 172.21.15.72:0/109411535 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:06.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dceb7700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d8063350 0x7fd3d80a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:06.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3cffff700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d80bc930 0x7fd3d814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:06.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dd6b8700 1 -- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d80bc930 msgr2=0x7fd3d814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:06.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dd6b8700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d80bc930 0x7fd3d814b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:06.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dd6b8700 1 -- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d8063350 msgr2=0x7fd3d80a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:06.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dd6b8700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d8063350 0x7fd3d80a7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:06.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dd6b8700 1 -- 172.21.15.72:0/109411535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d00092a0 con 0x7fd3d81498d0 2023-10-18T18:03:06.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3dd6b8700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d81498d0 0x7fd3d8146530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd3d0009d80 tx=0x7fd3d0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:06.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ccff9700 1 -- 172.21.15.72:0/109411535 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000f9f0 con 0x7fd3d81498d0 2023-10-18T18:03:06.183 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ccff9700 1 -- 172.21.15.72:0/109411535 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d000fe80 con 0x7fd3d81498d0 2023-10-18T18:03:06.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d80a4b90 con 0x7fd3d81498d0 2023-10-18T18:03:06.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.172+0000 7fd3ccff9700 1 -- 172.21.15.72:0/109411535 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d001b690 con 0x7fd3d81498d0 2023-10-18T18:03:06.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.176+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d80a5150 con 0x7fd3d81498d0 2023-10-18T18:03:06.184 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.176+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d80a5670 con 0x7fd3d81498d0 2023-10-18T18:03:06.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.176+0000 7fd3ccff9700 1 -- 172.21.15.72:0/109411535 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd3d001ba00 con 0x7fd3d81498d0 2023-10-18T18:03:06.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.180+0000 7fd3ccff9700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd3c405a510 0x7fd3c405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:06.185 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.180+0000 7fd3ccff9700 1 -- 172.21.15.72:0/109411535 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fd3d0076390 con 0x7fd3d81498d0 2023-10-18T18:03:06.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.180+0000 7fd3dceb7700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd3c405a510 0x7fd3c405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:06.186 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.180+0000 7fd3dceb7700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd3c405a510 0x7fd3c405c9d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fd3c8005750 tx=0x7fd3c8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:06.192 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.184+0000 7fd3ccff9700 1 -- 172.21.15.72:0/109411535 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fd3d80a5670 con 0x7fd3d81498d0 2023-10-18T18:03:06.388 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.380+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd3d80a5670 con 0x7fd3d81498d0 2023-10-18T18:03:06.389 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.384+0000 7fd3ccff9700 1 -- 172.21.15.72:0/109411535 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fd3d80a5670 con 0x7fd3d81498d0 2023-10-18T18:03:06.389 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:06.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd3c405a510 msgr2=0x7fd3c405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:06.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd3c405a510 0x7fd3c405c9d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fd3c8005750 tx=0x7fd3c8009040).stop 2023-10-18T18:03:06.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d81498d0 msgr2=0x7fd3d8146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:06.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d81498d0 0x7fd3d8146530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd3d0009d80 tx=0x7fd3d0017000).stop 2023-10-18T18:03:06.395 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 shutdown_connections 2023-10-18T18:03:06.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd3c405a510 0x7fd3c405c9d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:06.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3d81498d0 0x7fd3d8146530 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:06.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd3d80bc930 0x7fd3d814b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:06.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 --2- 172.21.15.72:0/109411535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d8063350 0x7fd3d80a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:06.396 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 >> 172.21.15.72:0/109411535 conn(0x7fd3d8062ba0 msgr2=0x7fd3d80bdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:06.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 shutdown_connections 2023-10-18T18:03:06.397 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:06.388+0000 7fd3ddeb9700 1 -- 172.21.15.72:0/109411535 wait complete. 2023-10-18T18:03:06.415 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:07.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~277s 2023-10-18T18:03:07.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~222s 2023-10-18T18:03:07.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~177s 2023-10-18T18:03:07.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~203s 2023-10-18T18:03:07.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~190s 2023-10-18T18:03:07.850 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~236s 2023-10-18T18:03:09.416 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:09.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.576+0000 7fd5594db700 1 Processor -- start 2023-10-18T18:03:09.584 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.576+0000 7fd5594db700 1 -- start start 2023-10-18T18:03:09.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd5594db700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd5540bb770 0x7fd554068a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:09.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd5594db700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7fd554151ee0 0x7fd554157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:09.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd5594db700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5540685d0 con 0x7fd5540620d0 2023-10-18T18:03:09.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd5594db700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd554067d70 con 0x7fd5540bb770 2023-10-18T18:03:09.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd5594db700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd554158310 con 0x7fd554151ee0 2023-10-18T18:03:09.587 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd552ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd5540bb770 0x7fd554068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:09.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd552ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fd5540bb770 0x7fd554068a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:54680/0 (socket says 172.21.15.72:54680) 2023-10-18T18:03:09.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd552ffd700 1 -- 172.21.15.72:0/1288939646 learned_addr learned my addr 172.21.15.72:0/1288939646 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:09.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd54bfff700 1 --2- 172.21.15.72:0/1288939646 >> v2:172.21.15.179:3300/0 conn(0x7fd554151ee0 0x7fd554157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:09.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd552ffd700 1 -- 172.21.15.72:0/1288939646 >> v2:172.21.15.179:3300/0 conn(0x7fd554151ee0 msgr2=0x7fd554157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:03:09.588 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd552ffd700 1 --2- 172.21.15.72:0/1288939646 >> v2:172.21.15.179:3300/0 conn(0x7fd554151ee0 0x7fd554157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:09.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd552ffd700 1 -- 172.21.15.72:0/1288939646 >> v1:172.21.15.72:6789/0 conn(0x7fd5540620d0 legacy=0x7fd5540624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:09.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd552ffd700 1 -- 172.21.15.72:0/1288939646 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd554152300 con 0x7fd5540bb770 2023-10-18T18:03:09.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd54bfff700 1 --2- 172.21.15.72:0/1288939646 >> v2:172.21.15.179:3300/0 conn(0x7fd554151ee0 0x7fd554157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:03:09.589 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd552ffd700 1 --2- 172.21.15.72:0/1288939646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5540bb770 0x7fd554068a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fd5540a8330 tx=0x7fd5440090f0).ready entity=mon.0 client_cookie=58f79e94edbb77ed server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:09.590 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd5527fc700 1 -- 172.21.15.72:0/1288939646 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54400b520 con 0x7fd5540bb770 2023-10-18T18:03:09.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd5527fc700 1 -- 172.21.15.72:0/1288939646 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54400b680 con 0x7fd5540bb770 2023-10-18T18:03:09.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.580+0000 7fd5527fc700 1 -- 172.21.15.72:0/1288939646 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54400ba00 con 0x7fd5540bb770 2023-10-18T18:03:09.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 -- 172.21.15.72:0/1288939646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5540bb770 msgr2=0x7fd554068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:09.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 --2- 172.21.15.72:0/1288939646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5540bb770 0x7fd554068a60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fd5540a8330 tx=0x7fd5440090f0).stop 2023-10-18T18:03:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 -- 172.21.15.72:0/1288939646 shutdown_connections 2023-10-18T18:03:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 --2- 172.21.15.72:0/1288939646 >> v2:172.21.15.179:3300/0 conn(0x7fd554151ee0 0x7fd554157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 --2- 172.21.15.72:0/1288939646 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd5540bb770 0x7fd554068a60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 -- 172.21.15.72:0/1288939646 >> 172.21.15.72:0/1288939646 conn(0x7fd5540658b0 msgr2=0x7fd554065c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:09.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 -- 172.21.15.72:0/1288939646 shutdown_connections 2023-10-18T18:03:09.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 -- 172.21.15.72:0/1288939646 wait complete. 2023-10-18T18:03:09.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 Processor -- start 2023-10-18T18:03:09.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.584+0000 7fd5594db700 1 -- start start 2023-10-18T18:03:09.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5594db700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5540620d0 0x7fd5541654b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:09.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5594db700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd5540bb770 0x7fd554164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:09.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5594db700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd554151ee0 0x7fd554164760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:09.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5594db700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540bf4c0 con 0x7fd554151ee0 2023-10-18T18:03:09.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5594db700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540bee10 con 0x7fd5540620d0 2023-10-18T18:03:09.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5594db700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540a0bd0 con 0x7fd5540bb770 2023-10-18T18:03:09.596 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5537fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd554151ee0 0x7fd554164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:09.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd552ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5540620d0 0x7fd5541654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:09.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd54bfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd5540bb770 0x7fd554164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:09.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd552ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5540620d0 0x7fd5541654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:54986/0 (socket says 172.21.15.72:54986) 2023-10-18T18:03:09.597 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd552ffd700 1 -- 172.21.15.72:0/416877109 learned_addr learned my addr 172.21.15.72:0/416877109 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:09.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5537fe700 1 -- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd5540bb770 msgr2=0x7fd554164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:09.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5537fe700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd5540bb770 0x7fd554164270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:09.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5537fe700 1 -- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5540620d0 msgr2=0x7fd5541654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:09.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5537fe700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5540620d0 0x7fd5541654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:09.598 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5537fe700 1 -- 172.21.15.72:0/416877109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c0092a0 con 0x7fd554151ee0 2023-10-18T18:03:09.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5537fe700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd554151ee0 0x7fd554164760 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fd54c00c790 tx=0x7fd54c001460).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:09.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd54affd700 1 -- 172.21.15.72:0/416877109 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00f470 con 0x7fd554151ee0 2023-10-18T18:03:09.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd54affd700 1 -- 172.21.15.72:0/416877109 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c00f5d0 con 0x7fd554151ee0 2023-10-18T18:03:09.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd544009370 con 0x7fd554151ee0 2023-10-18T18:03:09.599 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd54affd700 1 -- 172.21.15.72:0/416877109 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c00f950 con 0x7fd554151ee0 2023-10-18T18:03:09.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.588+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5540bdd70 con 0x7fd554151ee0 2023-10-18T18:03:09.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.592+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd554046f00 con 0x7fd554151ee0 2023-10-18T18:03:09.600 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.592+0000 7fd54affd700 1 -- 172.21.15.72:0/416877109 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fd54c00fd40 con 0x7fd554151ee0 2023-10-18T18:03:09.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.592+0000 7fd54affd700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd53405a540 0x7fd53405ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:09.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.592+0000 7fd54affd700 1 -- 172.21.15.72:0/416877109 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fd54c073f50 con 0x7fd554151ee0 2023-10-18T18:03:09.601 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.592+0000 7fd552ffd700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd53405a540 0x7fd53405ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:09.602 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.596+0000 7fd552ffd700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd53405a540 0x7fd53405ca00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd54400b2e0 tx=0x7fd544017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:09.607 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.600+0000 7fd54affd700 1 -- 172.21.15.72:0/416877109 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fd54c0474f0 con 0x7fd554151ee0 2023-10-18T18:03:09.801 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.796+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd5540bf880 con 0x7fd554151ee0 2023-10-18T18:03:09.802 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.796+0000 7fd54affd700 1 -- 172.21.15.72:0/416877109 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fd54c04ab10 con 0x7fd554151ee0 2023-10-18T18:03:09.802 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:09.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd53405a540 msgr2=0x7fd53405ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:09.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd53405a540 0x7fd53405ca00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd54400b2e0 tx=0x7fd544017000).stop 2023-10-18T18:03:09.806 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd554151ee0 msgr2=0x7fd554164760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:09.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd554151ee0 0x7fd554164760 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fd54c00c790 tx=0x7fd54c001460).stop 2023-10-18T18:03:09.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 shutdown_connections 2023-10-18T18:03:09.807 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fd53405a540 0x7fd53405ca00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:09.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd554151ee0 0x7fd554164760 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:09.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fd5540bb770 0x7fd554164270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:09.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 --2- 172.21.15.72:0/416877109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5540620d0 0x7fd5541654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:09.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 >> 172.21.15.72:0/416877109 conn(0x7fd5540658b0 msgr2=0x7fd5540badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:09.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 shutdown_connections 2023-10-18T18:03:09.808 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:09.800+0000 7fd5594db700 1 -- 172.21.15.72:0/416877109 wait complete. 2023-10-18T18:03:09.827 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:12.829 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:12.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.979+0000 7ffbc1cdb700 1 Processor -- start 2023-10-18T18:03:12.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.983+0000 7ffbc1cdb700 1 -- start start 2023-10-18T18:03:12.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.983+0000 7ffbc1cdb700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffbbc0beac0 0x7ffbbc0beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:12.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.983+0000 7ffbc1cdb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc066150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:12.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc0bd9e0 con 0x7ffbbc066640 2023-10-18T18:03:12.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbbc150fb0 con 0x7ffbbc0beac0 2023-10-18T18:03:12.993 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0bcaf0 con 0x7ffbbc150b90 2023-10-18T18:03:12.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbb2ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc066150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:12.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbb2ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc066150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:53594/0 (socket says 172.21.15.72:53594) 2023-10-18T18:03:12.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbb2ffd700 1 -- 172.21.15.72:0/2395472644 learned_addr learned my addr 172.21.15.72:0/2395472644 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:12.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbbb7fe700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7ffbbc0beac0 0x7ffbbc0beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:12.995 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbbbfff700 1 --1- 172.21.15.72:0/2395472644 >> v1:172.21.15.179:6789/0 conn(0x7ffbbc066640 0x7ffbbc067e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:43848/0 (socket says 172.21.15.72:43848) 2023-10-18T18:03:12.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbb2ffd700 1 -- 172.21.15.72:0/2395472644 >> v2:172.21.15.179:3300/0 conn(0x7ffbbc0beac0 msgr2=0x7ffbbc0beea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:03:12.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbb2ffd700 1 --2- 172.21.15.72:0/2395472644 >> v2:172.21.15.179:3300/0 conn(0x7ffbbc0beac0 0x7ffbbc0beea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:12.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbb2ffd700 1 -- 172.21.15.72:0/2395472644 >> v1:172.21.15.179:6789/0 conn(0x7ffbbc066640 legacy=0x7ffbbc067e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:12.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbb2ffd700 1 -- 172.21.15.72:0/2395472644 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc0bcdd0 con 0x7ffbbc150b90 2023-10-18T18:03:12.996 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbbaffd700 1 -- 172.21.15.72:0/2395472644 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098115224 0 0) 0x7ffbbc0bd9e0 con 0x7ffbbc066640 2023-10-18T18:03:12.997 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbb2ffd700 1 --2- 172.21.15.72:0/2395472644 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc066150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7ffba4006fd0 tx=0x7ffba40090f0).ready entity=mon.2 client_cookie=203ade55f0caa992 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbbaffd700 1 -- 172.21.15.72:0/2395472644 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400b610 con 0x7ffbbc150b90 2023-10-18T18:03:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbbaffd700 1 -- 172.21.15.72:0/2395472644 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba400b770 con 0x7ffbbc150b90 2023-10-18T18:03:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbbaffd700 1 -- 172.21.15.72:0/2395472644 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400baf0 con 0x7ffbbc150b90 2023-10-18T18:03:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/2395472644 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 msgr2=0x7ffbbc066150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:12.998 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 --2- 172.21.15.72:0/2395472644 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc066150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7ffba4006fd0 tx=0x7ffba40090f0).stop 2023-10-18T18:03:12.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/2395472644 shutdown_connections 2023-10-18T18:03:12.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 --2- 172.21.15.72:0/2395472644 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc066150 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:12.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 --2- 172.21.15.72:0/2395472644 >> v2:172.21.15.179:3300/0 conn(0x7ffbbc0beac0 0x7ffbbc0beea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:12.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/2395472644 >> 172.21.15.72:0/2395472644 conn(0x7ffbbc0620d0 msgr2=0x7ffbbc0624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:12.999 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/2395472644 shutdown_connections 2023-10-18T18:03:13.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.987+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/2395472644 wait complete. 2023-10-18T18:03:13.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.991+0000 7ffbc1cdb700 1 Processor -- start 2023-10-18T18:03:13.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.991+0000 7ffbc1cdb700 1 -- start start 2023-10-18T18:03:13.001 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbc1cdb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbbc066640 0x7ffbbc14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbc1cdb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffbbc0beac0 0x7ffbbc14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbc1cdb700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc14b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbc1cdb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc146920 con 0x7ffbbc0beac0 2023-10-18T18:03:13.002 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbc1cdb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc14b350 con 0x7ffbbc066640 2023-10-18T18:03:13.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbc1cdb700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc14db00 con 0x7ffbbc150b90 2023-10-18T18:03:13.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbbc066640 0x7ffbbc14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:13.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbbfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc14b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:13.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbb2ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffbbc0beac0 0x7ffbbc14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:13.003 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbbfff700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc14b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3301/0 says I am v2:172.21.15.72:53598/0 (socket says 172.21.15.72:53598) 2023-10-18T18:03:13.004 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbbfff700 1 -- 172.21.15.72:0/3152071101 learned_addr learned my addr 172.21.15.72:0/3152071101 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:13.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbbfff700 1 -- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbbc066640 msgr2=0x7ffbbc14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:13.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbbfff700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbbc066640 0x7ffbbc14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:13.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbbfff700 1 -- 172.21.15.72:0/3152071101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffbbc0beac0 msgr2=0x7ffbbc14d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:13.005 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbbfff700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffbbc0beac0 0x7ffbbc14d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:13.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbbfff700 1 -- 172.21.15.72:0/3152071101 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4009370 con 0x7ffbbc150b90 2023-10-18T18:03:13.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbb2ffd700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffbbc0beac0 0x7ffbbc14d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:03:13.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbb7fe700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbbc066640 0x7ffbbc14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:03:13.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbbbfff700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc14b7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7ffbb400c7e0 tx=0x7ffbb4007f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:13.006 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbb3fff700 1 -- 172.21.15.72:0/3152071101 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4004c40 con 0x7ffbbc150b90 2023-10-18T18:03:13.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbb3fff700 1 -- 172.21.15.72:0/3152071101 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4004da0 con 0x7ffbbc150b90 2023-10-18T18:03:13.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc14aa40 con 0x7ffbbc150b90 2023-10-18T18:03:13.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbb3fff700 1 -- 172.21.15.72:0/3152071101 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40073a0 con 0x7ffbbc150b90 2023-10-18T18:03:13.007 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.995+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc14afe0 con 0x7ffbbc150b90 2023-10-18T18:03:13.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.999+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc06c190 con 0x7ffbbc150b90 2023-10-18T18:03:13.008 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:12.999+0000 7ffbb3fff700 1 -- 172.21.15.72:0/3152071101 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7ffbb4007750 con 0x7ffbbc150b90 2023-10-18T18:03:13.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.003+0000 7ffbb3fff700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffba005a510 0x7ffba005c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:13.009 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.003+0000 7ffbb3fff700 1 -- 172.21.15.72:0/3152071101 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7ffbb40745b0 con 0x7ffbbc150b90 2023-10-18T18:03:13.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.003+0000 7ffbbb7fe700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffba005a510 0x7ffba005c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:13.010 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.003+0000 7ffbbb7fe700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffba005a510 0x7ffba005c9d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ffbac011310 tx=0x7ffbac009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:13.020 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.011+0000 7ffbb3fff700 1 -- 172.21.15.72:0/3152071101 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7ffbb4047b00 con 0x7ffbbc150b90 2023-10-18T18:03:13.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.207+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffbbc14b490 con 0x7ffbbc150b90 2023-10-18T18:03:13.216 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.207+0000 7ffbb3fff700 1 -- 172.21.15.72:0/3152071101 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7ffbb4019070 con 0x7ffbbc150b90 2023-10-18T18:03:13.217 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:13.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffba005a510 msgr2=0x7ffba005c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:13.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffba005a510 0x7ffba005c9d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7ffbac011310 tx=0x7ffbac009060).stop 2023-10-18T18:03:13.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 msgr2=0x7ffbbc14b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:13.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc14b7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7ffbb400c7e0 tx=0x7ffbb4007f60).stop 2023-10-18T18:03:13.222 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 shutdown_connections 2023-10-18T18:03:13.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7ffba005a510 0x7ffba005c9d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:13.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7ffbbc150b90 0x7ffbbc14b7e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:13.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffbbc0beac0 0x7ffbbc14d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:13.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 --2- 172.21.15.72:0/3152071101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbbc066640 0x7ffbbc14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:13.223 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 >> 172.21.15.72:0/3152071101 conn(0x7ffbbc0620d0 msgr2=0x7ffbbc0c1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:13.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 shutdown_connections 2023-10-18T18:03:13.224 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:13.215+0000 7ffbc1cdb700 1 -- 172.21.15.72:0/3152071101 wait complete. 2023-10-18T18:03:13.242 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:14.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~283s 2023-10-18T18:03:14.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~229s 2023-10-18T18:03:14.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~183s 2023-10-18T18:03:14.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~209s 2023-10-18T18:03:14.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~196s 2023-10-18T18:03:14.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~242s 2023-10-18T18:03:16.244 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:16.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.399+0000 7f5062b79700 1 Processor -- start 2023-10-18T18:03:16.408 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.399+0000 7f5062b79700 1 -- start start 2023-10-18T18:03:16.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5062b79700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c0658a0 0x7f505c065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:16.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5062b79700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f505c0630e0 0x7f505c14cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:16.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5062b79700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c06a760 con 0x7f505c069f20 2023-10-18T18:03:16.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5062b79700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0c2d40 con 0x7f505c0630e0 2023-10-18T18:03:16.410 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5062b79700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c150bf0 con 0x7f505c0658a0 2023-10-18T18:03:16.411 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5061376700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7f505c069f20 0x7f505c0c2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:45100/0 (socket says 172.21.15.72:45100) 2023-10-18T18:03:16.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5061376700 1 -- 172.21.15.72:0/1822149817 learned_addr learned my addr 172.21.15.72:0/1822149817 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:03:16.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5061b77700 1 --2- 172.21.15.72:0/1822149817 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c0658a0 0x7f505c065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:16.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5062378700 1 --2- 172.21.15.72:0/1822149817 >> v2:172.21.15.179:3300/0 conn(0x7f505c0630e0 0x7f505c14cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:16.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5060b75700 1 -- 172.21.15.72:0/1822149817 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66709448 0 0) 0x7f505c06a760 con 0x7f505c069f20 2023-10-18T18:03:16.412 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5060b75700 1 -- 172.21.15.72:0/1822149817 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50440033f0 con 0x7f505c069f20 2023-10-18T18:03:16.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5061b77700 1 -- 172.21.15.72:0/1822149817 >> v2:172.21.15.179:3300/0 conn(0x7f505c0630e0 msgr2=0x7f505c14cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:03:16.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5061b77700 1 --2- 172.21.15.72:0/1822149817 >> v2:172.21.15.179:3300/0 conn(0x7f505c0630e0 0x7f505c14cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:16.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5061b77700 1 -- 172.21.15.72:0/1822149817 >> v1:172.21.15.72:6789/0 conn(0x7f505c069f20 legacy=0x7f505c0c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:16.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5061b77700 1 -- 172.21.15.72:0/1822149817 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c06a480 con 0x7f505c0658a0 2023-10-18T18:03:16.413 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5061b77700 1 --2- 172.21.15.72:0/1822149817 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c0658a0 0x7f505c065c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f504c009760 tx=0x7f504c009060).ready entity=mon.2 client_cookie=50e6c27078351f9a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:16.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5060b75700 1 -- 172.21.15.72:0/1822149817 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c00f510 con 0x7f505c0658a0 2023-10-18T18:03:16.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5060b75700 1 -- 172.21.15.72:0/1822149817 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c00f670 con 0x7f505c0658a0 2023-10-18T18:03:16.414 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5060b75700 1 -- 172.21.15.72:0/1822149817 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c00f9f0 con 0x7f505c0658a0 2023-10-18T18:03:16.415 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5062b79700 1 -- 172.21.15.72:0/1822149817 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c0658a0 msgr2=0x7f505c065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:16.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.403+0000 7f5062b79700 1 --2- 172.21.15.72:0/1822149817 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c0658a0 0x7f505c065c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f504c009760 tx=0x7f504c009060).stop 2023-10-18T18:03:16.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.407+0000 7f5062b79700 1 -- 172.21.15.72:0/1822149817 shutdown_connections 2023-10-18T18:03:16.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.407+0000 7f5062b79700 1 --2- 172.21.15.72:0/1822149817 >> v2:172.21.15.179:3300/0 conn(0x7f505c0630e0 0x7f505c14cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:16.416 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.407+0000 7f5062b79700 1 --2- 172.21.15.72:0/1822149817 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c0658a0 0x7f505c065c80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:16.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.407+0000 7f5062b79700 1 -- 172.21.15.72:0/1822149817 >> 172.21.15.72:0/1822149817 conn(0x7f505c062ba0 msgr2=0x7f505c0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:16.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.407+0000 7f5062b79700 1 -- 172.21.15.72:0/1822149817 shutdown_connections 2023-10-18T18:03:16.417 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.407+0000 7f5062b79700 1 -- 172.21.15.72:0/1822149817 wait complete. 2023-10-18T18:03:16.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.407+0000 7f5062b79700 1 Processor -- start 2023-10-18T18:03:16.418 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5062b79700 1 -- start start 2023-10-18T18:03:16.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5062b79700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f505c0630e0 0x7f505c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:16.419 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5062b79700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f505c0658a0 0x7f505c14b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:16.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5062b79700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c069f20 0x7f505c155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:16.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5062b79700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c14cab0 con 0x7f505c0658a0 2023-10-18T18:03:16.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5062b79700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0b5c90 con 0x7f505c0630e0 2023-10-18T18:03:16.420 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5062b79700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c157980 con 0x7f505c069f20 2023-10-18T18:03:16.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061376700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f505c0658a0 0x7f505c14b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:16.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061b77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f505c0630e0 0x7f505c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:16.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061b77700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f505c0630e0 0x7f505c155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:40642/0 (socket says 172.21.15.72:40642) 2023-10-18T18:03:16.421 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061b77700 1 -- 172.21.15.72:0/12568389 learned_addr learned my addr 172.21.15.72:0/12568389 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:16.422 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5062378700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c069f20 0x7f505c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:16.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061b77700 1 -- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c069f20 msgr2=0x7f505c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:16.423 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061b77700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c069f20 0x7f505c155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:16.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061b77700 1 -- 172.21.15.72:0/12568389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f505c0658a0 msgr2=0x7f505c14b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:16.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061b77700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f505c0658a0 0x7f505c14b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:16.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061b77700 1 -- 172.21.15.72:0/12568389 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c0092a0 con 0x7f505c0630e0 2023-10-18T18:03:16.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061376700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f505c0658a0 0x7f505c14b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:03:16.424 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5061b77700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f505c0630e0 0x7f505c155a40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f504c009d50 tx=0x7f504c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:16.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.411+0000 7f5059ffb700 1 -- 172.21.15.72:0/12568389 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c00f510 con 0x7f505c0630e0 2023-10-18T18:03:16.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.415+0000 7f5059ffb700 1 -- 172.21.15.72:0/12568389 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c00f7e0 con 0x7f505c0630e0 2023-10-18T18:03:16.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.415+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c14bc00 con 0x7f505c0630e0 2023-10-18T18:03:16.425 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.415+0000 7f5059ffb700 1 -- 172.21.15.72:0/12568389 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c01b440 con 0x7f505c0630e0 2023-10-18T18:03:16.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.415+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c156040 con 0x7f505c0630e0 2023-10-18T18:03:16.426 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.415+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c1577a0 con 0x7f505c0630e0 2023-10-18T18:03:16.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.419+0000 7f5059ffb700 1 -- 172.21.15.72:0/12568389 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f504c01b7b0 con 0x7f505c0630e0 2023-10-18T18:03:16.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.419+0000 7f5059ffb700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f504805a4e0 0x7f504805c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:16.427 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.419+0000 7f5059ffb700 1 -- 172.21.15.72:0/12568389 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f504c076be0 con 0x7f505c0630e0 2023-10-18T18:03:16.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.419+0000 7f5061376700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f504805a4e0 0x7f504805c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:16.428 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.419+0000 7f5061376700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f504805a4e0 0x7f504805c9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f5050005750 tx=0x7f5050009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:16.433 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.427+0000 7f5059ffb700 1 -- 172.21.15.72:0/12568389 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f504c04a1b0 con 0x7f505c0630e0 2023-10-18T18:03:16.640 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.631+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f505c046f00 con 0x7f505c0630e0 2023-10-18T18:03:16.645 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.639+0000 7f5059ffb700 1 -- 172.21.15.72:0/12568389 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f504c04d7d0 con 0x7f505c0630e0 2023-10-18T18:03:16.646 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:16.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f504805a4e0 msgr2=0x7f504805c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:16.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f504805a4e0 0x7f504805c9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f5050005750 tx=0x7f5050009040).stop 2023-10-18T18:03:16.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f505c0630e0 msgr2=0x7f505c155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:16.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f505c0630e0 0x7f505c155a40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f504c009d50 tx=0x7f504c017000).stop 2023-10-18T18:03:16.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 shutdown_connections 2023-10-18T18:03:16.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f504805a4e0 0x7f504805c9a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:16.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f505c069f20 0x7f505c155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:16.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f505c0658a0 0x7f505c14b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:16.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 --2- 172.21.15.72:0/12568389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f505c0630e0 0x7f505c155a40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:16.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 >> 172.21.15.72:0/12568389 conn(0x7f505c062ba0 msgr2=0x7f505c0c14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:16.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 shutdown_connections 2023-10-18T18:03:16.653 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:16.643+0000 7f5062b79700 1 -- 172.21.15.72:0/12568389 wait complete. 2023-10-18T18:03:16.671 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:19.673 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:19.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.839+0000 7f3370df9700 1 Processor -- start 2023-10-18T18:03:19.848 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.839+0000 7f3370df9700 1 -- start start 2023-10-18T18:03:19.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f3370df9700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f336c0c2bf0 0x7f336c065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:19.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f3370df9700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336c0be2e0 con 0x7f336c0658a0 2023-10-18T18:03:19.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f3370df9700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336c0be770 con 0x7f336c0bd1c0 2023-10-18T18:03:19.850 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f3370df9700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336c0bda80 con 0x7f336c0c2bf0 2023-10-18T18:03:19.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336bfff700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f336c0bd1c0 0x7f336c14ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:41702/0 (socket says 172.21.15.72:41702) 2023-10-18T18:03:19.851 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336bfff700 1 -- 172.21.15.72:0/2190115965 learned_addr learned my addr 172.21.15.72:0/2190115965 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:03:19.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336affd700 1 --2- 172.21.15.72:0/2190115965 >> v2:172.21.15.72:3300/0 conn(0x7f336c0c2bf0 0x7f336c065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:19.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336b7fe700 1 --1- 172.21.15.72:0/2190115965 >> v1:172.21.15.72:6789/0 conn(0x7f336c0658a0 0x7f336c0630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:34090/0 (socket says 172.21.15.72:34090) 2023-10-18T18:03:19.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336a7fc700 1 -- 172.21.15.72:0/2190115965 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678540422 0 0) 0x7f336c0be770 con 0x7f336c0bd1c0 2023-10-18T18:03:19.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336a7fc700 1 -- 172.21.15.72:0/2190115965 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33540033f0 con 0x7f336c0bd1c0 2023-10-18T18:03:19.852 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336affd700 1 -- 172.21.15.72:0/2190115965 >> v1:172.21.15.179:6789/0 conn(0x7f336c0bd1c0 legacy=0x7f336c14ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:19.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336affd700 1 -- 172.21.15.72:0/2190115965 >> v1:172.21.15.72:6789/0 conn(0x7f336c0658a0 legacy=0x7f336c0630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:19.853 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336affd700 1 -- 172.21.15.72:0/2190115965 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c0b9580 con 0x7f336c0c2bf0 2023-10-18T18:03:19.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336affd700 1 --2- 172.21.15.72:0/2190115965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f336c0c2bf0 0x7f336c065c80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f3360009760 tx=0x7f3360009060).ready entity=mon.0 client_cookie=7052092e726b2e66 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:19.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336a7fc700 1 -- 172.21.15.72:0/2190115965 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336000f510 con 0x7f336c0c2bf0 2023-10-18T18:03:19.854 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336a7fc700 1 -- 172.21.15.72:0/2190115965 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336000f670 con 0x7f336c0c2bf0 2023-10-18T18:03:19.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.843+0000 7f336a7fc700 1 -- 172.21.15.72:0/2190115965 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336000f9f0 con 0x7f336c0c2bf0 2023-10-18T18:03:19.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 -- 172.21.15.72:0/2190115965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f336c0c2bf0 msgr2=0x7f336c065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:19.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 --2- 172.21.15.72:0/2190115965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f336c0c2bf0 0x7f336c065c80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f3360009760 tx=0x7f3360009060).stop 2023-10-18T18:03:19.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 -- 172.21.15.72:0/2190115965 shutdown_connections 2023-10-18T18:03:19.855 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 --2- 172.21.15.72:0/2190115965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f336c0c2bf0 0x7f336c065c80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:19.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 -- 172.21.15.72:0/2190115965 >> 172.21.15.72:0/2190115965 conn(0x7f336c062ba0 msgr2=0x7f336c0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:19.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 -- 172.21.15.72:0/2190115965 shutdown_connections 2023-10-18T18:03:19.856 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 -- 172.21.15.72:0/2190115965 wait complete. 2023-10-18T18:03:19.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 Processor -- start 2023-10-18T18:03:19.857 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 -- start start 2023-10-18T18:03:19.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.847+0000 7f3370df9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f336c0658a0 0x7f336c1637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:19.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f3370df9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f336c0bd1c0 0x7f336c1504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:19.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f3370df9700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f336c0c2bf0 0x7f336c149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:19.858 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f3370df9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c068510 con 0x7f336c0bd1c0 2023-10-18T18:03:19.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f3370df9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0b6080 con 0x7f336c0658a0 2023-10-18T18:03:19.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f3370df9700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c163cc0 con 0x7f336c0c2bf0 2023-10-18T18:03:19.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f336c0658a0 0x7f336c1637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:19.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f336c0658a0 0x7f336c1637d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:46316/0 (socket says 172.21.15.72:46316) 2023-10-18T18:03:19.859 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f336c0bd1c0 0x7f336c1504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:19.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336b7fe700 1 -- 172.21.15.72:0/1862084091 learned_addr learned my addr 172.21.15.72:0/1862084091 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:19.860 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336bfff700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f336c0c2bf0 0x7f336c149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:19.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336b7fe700 1 -- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f336c0c2bf0 msgr2=0x7f336c149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:19.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336b7fe700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f336c0c2bf0 0x7f336c149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:19.861 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336b7fe700 1 -- 172.21.15.72:0/1862084091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f336c0bd1c0 msgr2=0x7f336c1504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:19.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336b7fe700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f336c0bd1c0 0x7f336c1504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:19.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336b7fe700 1 -- 172.21.15.72:0/1862084091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600092a0 con 0x7f336c0658a0 2023-10-18T18:03:19.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f336b7fe700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f336c0658a0 0x7f336c1637d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f335c00c490 tx=0x7f335c007df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:19.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f33537fe700 1 -- 172.21.15.72:0/1862084091 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c00ce30 con 0x7f336c0658a0 2023-10-18T18:03:19.862 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f33537fe700 1 -- 172.21.15.72:0/1862084091 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c0072e0 con 0x7f336c0658a0 2023-10-18T18:03:19.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c14b790 con 0x7f336c0658a0 2023-10-18T18:03:19.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f33537fe700 1 -- 172.21.15.72:0/1862084091 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c0056e0 con 0x7f336c0658a0 2023-10-18T18:03:19.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c14ba30 con 0x7f336c0658a0 2023-10-18T18:03:19.863 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.851+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336c046f00 con 0x7f336c0658a0 2023-10-18T18:03:19.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.855+0000 7f33537fe700 1 -- 172.21.15.72:0/1862084091 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f335c005ad0 con 0x7f336c0658a0 2023-10-18T18:03:19.864 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.855+0000 7f33537fe700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f335805a510 0x7f335805c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:19.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.855+0000 7f33537fe700 1 -- 172.21.15.72:0/1862084091 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f335c074100 con 0x7f336c0658a0 2023-10-18T18:03:19.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.855+0000 7f336affd700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f335805a510 0x7f335805c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:19.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.855+0000 7f336affd700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f335805a510 0x7f335805c9d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f336c153ef0 tx=0x7f3360017000).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:19.873 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:19.863+0000 7f33537fe700 1 -- 172.21.15.72:0/1862084091 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f335c047650 con 0x7f336c0658a0 2023-10-18T18:03:20.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.063+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f336c0a8050 con 0x7f336c0658a0 2023-10-18T18:03:20.071 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.063+0000 7f33537fe700 1 -- 172.21.15.72:0/1862084091 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f335c04ac70 con 0x7f336c0658a0 2023-10-18T18:03:20.072 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:20.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f335805a510 msgr2=0x7f335805c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:20.074 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f335805a510 0x7f335805c9d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f336c153ef0 tx=0x7f3360017000).stop 2023-10-18T18:03:20.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f336c0658a0 msgr2=0x7f336c1637d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:20.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f336c0658a0 0x7f336c1637d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f335c00c490 tx=0x7f335c007df0).stop 2023-10-18T18:03:20.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 shutdown_connections 2023-10-18T18:03:20.075 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f335805a510 0x7f335805c9d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:20.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f336c0c2bf0 0x7f336c149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:20.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f336c0bd1c0 0x7f336c1504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:20.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 --2- 172.21.15.72:0/1862084091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f336c0658a0 0x7f336c1637d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:20.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 >> 172.21.15.72:0/1862084091 conn(0x7f336c062ba0 msgr2=0x7f336c153380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:20.076 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 shutdown_connections 2023-10-18T18:03:20.077 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:20.067+0000 7f3370df9700 1 -- 172.21.15.72:0/1862084091 wait complete. 2023-10-18T18:03:20.095 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:20.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~289s 2023-10-18T18:03:20.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~235s 2023-10-18T18:03:20.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~189s 2023-10-18T18:03:20.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~215s 2023-10-18T18:03:20.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~202s 2023-10-18T18:03:20.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~248s 2023-10-18T18:03:23.096 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:23.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcac5369700 1 Processor -- start 2023-10-18T18:03:23.265 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcac5369700 1 -- start start 2023-10-18T18:03:23.267 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcac5369700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac0067ff0 0x7fcac00683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:23.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcac5369700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7fcac0065e30 0x7fcac00bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:23.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcac5369700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac0066900 con 0x7fcac00bca50 2023-10-18T18:03:23.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcac5369700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac0068a50 con 0x7fcac0065e30 2023-10-18T18:03:23.268 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcac5369700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00659a0 con 0x7fcac0067ff0 2023-10-18T18:03:23.269 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcabf7fe700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7fcac00bca50 0x7fcac0150890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:41706/0 (socket says 172.21.15.72:41706) 2023-10-18T18:03:23.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcabf7fe700 1 -- 172.21.15.72:0/3289385624 learned_addr learned my addr 172.21.15.72:0/3289385624 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:03:23.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac0067ff0 0x7fcac00683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:23.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcab67fc700 1 --2- 172.21.15.72:0/3289385624 >> v2:172.21.15.72:3300/0 conn(0x7fcac0065e30 0x7fcac00bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:23.270 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcabe7fc700 1 -- 172.21.15.72:0/3289385624 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084556676 0 0) 0x7fcac0066900 con 0x7fcac00bca50 2023-10-18T18:03:23.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcabe7fc700 1 -- 172.21.15.72:0/3289385624 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaac0033f0 con 0x7fcac00bca50 2023-10-18T18:03:23.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.259+0000 7fcabeffd700 1 -- 172.21.15.72:0/3289385624 >> v2:172.21.15.72:3300/0 conn(0x7fcac0065e30 msgr2=0x7fcac00bdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:03:23.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcabeffd700 1 --2- 172.21.15.72:0/3289385624 >> v2:172.21.15.72:3300/0 conn(0x7fcac0065e30 0x7fcac00bdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:23.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcabeffd700 1 -- 172.21.15.72:0/3289385624 >> v1:172.21.15.179:6789/0 conn(0x7fcac00bca50 legacy=0x7fcac0150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:23.271 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcabeffd700 1 -- 172.21.15.72:0/3289385624 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0066620 con 0x7fcac0067ff0 2023-10-18T18:03:23.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcabeffd700 1 --2- 172.21.15.72:0/3289385624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac0067ff0 0x7fcac00683d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fcab0009e90 tx=0x7fcab0009780).ready entity=mon.2 client_cookie=94e21ebb3f1938f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:23.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcabe7fc700 1 -- 172.21.15.72:0/3289385624 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab000e070 con 0x7fcac0067ff0 2023-10-18T18:03:23.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcabe7fc700 1 -- 172.21.15.72:0/3289385624 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab000bb00 con 0x7fcac0067ff0 2023-10-18T18:03:23.272 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcabe7fc700 1 -- 172.21.15.72:0/3289385624 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab000be80 con 0x7fcac0067ff0 2023-10-18T18:03:23.273 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 -- 172.21.15.72:0/3289385624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac0067ff0 msgr2=0x7fcac00683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:23.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 --2- 172.21.15.72:0/3289385624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac0067ff0 0x7fcac00683d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fcab0009e90 tx=0x7fcab0009780).stop 2023-10-18T18:03:23.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 -- 172.21.15.72:0/3289385624 shutdown_connections 2023-10-18T18:03:23.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 --2- 172.21.15.72:0/3289385624 >> v2:172.21.15.72:3300/0 conn(0x7fcac0065e30 0x7fcac00bdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:23.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 --2- 172.21.15.72:0/3289385624 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac0067ff0 0x7fcac00683d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:23.274 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 -- 172.21.15.72:0/3289385624 >> 172.21.15.72:0/3289385624 conn(0x7fcac00620d0 msgr2=0x7fcac00624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:23.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 -- 172.21.15.72:0/3289385624 shutdown_connections 2023-10-18T18:03:23.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 -- 172.21.15.72:0/3289385624 wait complete. 2023-10-18T18:03:23.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 Processor -- start 2023-10-18T18:03:23.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.263+0000 7fcac5369700 1 -- start start 2023-10-18T18:03:23.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcac5369700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac0065e30 0x7fcac00c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:23.275 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcac5369700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcac0067ff0 0x7fcac0164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:23.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcac5369700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac00bca50 0x7fcac00b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:23.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcac5369700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00bbd00 con 0x7fcac0065e30 2023-10-18T18:03:23.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcac5369700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00b98f0 con 0x7fcac0067ff0 2023-10-18T18:03:23.276 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcac5369700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00c1ef0 con 0x7fcac00bca50 2023-10-18T18:03:23.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac0065e30 0x7fcac00c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:23.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabeffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac0065e30 0x7fcac00c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55276/0 (socket says 172.21.15.72:55276) 2023-10-18T18:03:23.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabeffd700 1 -- 172.21.15.72:0/4250760181 learned_addr learned my addr 172.21.15.72:0/4250760181 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:23.278 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcab67fc700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcac0067ff0 0x7fcac0164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:23.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabf7fe700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac00bca50 0x7fcac00b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:23.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabeffd700 1 -- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac00bca50 msgr2=0x7fcac00b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:23.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabeffd700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac00bca50 0x7fcac00b9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:23.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabeffd700 1 -- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcac0067ff0 msgr2=0x7fcac0164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:23.279 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabeffd700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcac0067ff0 0x7fcac0164670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:23.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabeffd700 1 -- 172.21.15.72:0/4250760181 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0009a20 con 0x7fcac0065e30 2023-10-18T18:03:23.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabf7fe700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac00bca50 0x7fcac00b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T18:03:23.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcabeffd700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac0065e30 0x7fcac00c1940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fcab000b610 tx=0x7fcab0009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:23.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcab77fe700 1 -- 172.21.15.72:0/4250760181 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab001d420 con 0x7fcac0065e30 2023-10-18T18:03:23.280 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcab77fe700 1 -- 172.21.15.72:0/4250760181 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab001d580 con 0x7fcac0065e30 2023-10-18T18:03:23.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcab77fe700 1 -- 172.21.15.72:0/4250760181 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab001d920 con 0x7fcac0065e30 2023-10-18T18:03:23.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac00c21d0 con 0x7fcac0065e30 2023-10-18T18:03:23.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.267+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac0163b90 con 0x7fcac0065e30 2023-10-18T18:03:23.281 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.271+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0067a70 con 0x7fcac0065e30 2023-10-18T18:03:23.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.271+0000 7fcab77fe700 1 -- 172.21.15.72:0/4250760181 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7fcab001dcd0 con 0x7fcac0065e30 2023-10-18T18:03:23.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.271+0000 7fcab77fe700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcaa005a540 0x7fcaa005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:23.282 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.271+0000 7fcab77fe700 1 -- 172.21.15.72:0/4250760181 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7fcab00766f0 con 0x7fcac0065e30 2023-10-18T18:03:23.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.271+0000 7fcab67fc700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcaa005a540 0x7fcaa005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:23.283 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.275+0000 7fcab67fc700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcaa005a540 0x7fcaa005ca00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fcac0146650 tx=0x7fcaa8009060).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:23.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.275+0000 7fcab77fe700 1 -- 172.21.15.72:0/4250760181 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7fcac0067a70 con 0x7fcac0065e30 2023-10-18T18:03:23.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.471+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcac0067a70 con 0x7fcac0065e30 2023-10-18T18:03:23.480 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.471+0000 7fcab77fe700 1 -- 172.21.15.72:0/4250760181 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7fcac0067a70 con 0x7fcac0065e30 2023-10-18T18:03:23.481 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:23.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.475+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcaa005a540 msgr2=0x7fcaa005ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:23.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcaa005a540 0x7fcaa005ca00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fcac0146650 tx=0x7fcaa8009060).stop 2023-10-18T18:03:23.485 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac0065e30 msgr2=0x7fcac00c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:23.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac0065e30 0x7fcac00c1940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fcab000b610 tx=0x7fcab0009780).stop 2023-10-18T18:03:23.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 shutdown_connections 2023-10-18T18:03:23.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7fcaa005a540 0x7fcaa005ca00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:23.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7fcac00bca50 0x7fcac00b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:23.486 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcac0067ff0 0x7fcac0164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:23.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 --2- 172.21.15.72:0/4250760181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcac0065e30 0x7fcac00c1940 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:23.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 >> 172.21.15.72:0/4250760181 conn(0x7fcac00620d0 msgr2=0x7fcac00be610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:23.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 shutdown_connections 2023-10-18T18:03:23.487 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:23.479+0000 7fcac5369700 1 -- 172.21.15.72:0/4250760181 wait complete. 2023-10-18T18:03:23.506 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:26.508 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:26.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.663+0000 7efd24907700 1 Processor -- start 2023-10-18T18:03:26.668 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.663+0000 7efd24907700 1 -- start start 2023-10-18T18:03:26.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd24907700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efd20151680 0x7efd20062380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:26.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd24907700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200bf550 0x7efd200bd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:26.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd24907700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd200c02d0 con 0x7efd200c2f30 2023-10-18T18:03:26.671 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd24907700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd200be9c0 con 0x7efd20151680 2023-10-18T18:03:26.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd24907700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd20062b80 con 0x7efd200bf550 2023-10-18T18:03:26.672 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1f7fe700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7efd20151680 0x7efd20062380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1effd700 1 --1- >> v1:172.21.15.72:6789/0 conn(0x7efd200c2f30 0x7efd20064d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.72:6789/0 says I am v1:172.21.15.72:34106/0 (socket says 172.21.15.72:34106) 2023-10-18T18:03:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1effd700 1 -- 172.21.15.72:0/2123568254 learned_addr learned my addr 172.21.15.72:0/2123568254 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:03:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1ffff700 1 --2- 172.21.15.72:0/2123568254 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200bf550 0x7efd200bd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:26.673 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1f7fe700 1 -- 172.21.15.72:0/2123568254 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200bf550 msgr2=0x7efd200bd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T18:03:26.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1f7fe700 1 --2- 172.21.15.72:0/2123568254 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200bf550 0x7efd200bd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:26.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1f7fe700 1 -- 172.21.15.72:0/2123568254 >> v1:172.21.15.72:6789/0 conn(0x7efd200c2f30 legacy=0x7efd20064d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:26.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1f7fe700 1 -- 172.21.15.72:0/2123568254 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd20154c10 con 0x7efd20151680 2023-10-18T18:03:26.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1ffff700 1 --2- 172.21.15.72:0/2123568254 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200bf550 0x7efd200bd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:03:26.674 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.667+0000 7efd1e7fc700 1 -- 172.21.15.72:0/2123568254 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485082000 0 0) 0x7efd200c02d0 con 0x7efd200c2f30 2023-10-18T18:03:26.675 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd1f7fe700 1 --2- 172.21.15.72:0/2123568254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd20151680 0x7efd20062380 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7efd10009760 tx=0x7efd10009060).ready entity=mon.0 client_cookie=3c9093887d4a4061 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:26.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd1e7fc700 1 -- 172.21.15.72:0/2123568254 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd100090d0 con 0x7efd20151680 2023-10-18T18:03:26.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd1e7fc700 1 -- 172.21.15.72:0/2123568254 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1000f510 con 0x7efd20151680 2023-10-18T18:03:26.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd1e7fc700 1 -- 172.21.15.72:0/2123568254 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1000f890 con 0x7efd20151680 2023-10-18T18:03:26.676 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 -- 172.21.15.72:0/2123568254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd20151680 msgr2=0x7efd20062380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:26.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 --2- 172.21.15.72:0/2123568254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd20151680 0x7efd20062380 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7efd10009760 tx=0x7efd10009060).stop 2023-10-18T18:03:26.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 -- 172.21.15.72:0/2123568254 shutdown_connections 2023-10-18T18:03:26.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 --2- 172.21.15.72:0/2123568254 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200bf550 0x7efd200bd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:26.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 --2- 172.21.15.72:0/2123568254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd20151680 0x7efd20062380 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:26.677 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 -- 172.21.15.72:0/2123568254 >> 172.21.15.72:0/2123568254 conn(0x7efd20069d80 msgr2=0x7efd2006a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:26.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 -- 172.21.15.72:0/2123568254 shutdown_connections 2023-10-18T18:03:26.678 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 -- 172.21.15.72:0/2123568254 wait complete. 2023-10-18T18:03:26.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 Processor -- start 2023-10-18T18:03:26.679 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.671+0000 7efd24907700 1 -- start start 2023-10-18T18:03:26.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd24907700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd200bf550 0x7efd20153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:26.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd24907700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200c2f30 0x7efd201551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:26.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd24907700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd20151680 0x7efd2015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:26.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd24907700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd200b6ef0 con 0x7efd20151680 2023-10-18T18:03:26.680 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd24907700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd201557f0 con 0x7efd200bf550 2023-10-18T18:03:26.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd24907700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd20155ad0 con 0x7efd200c2f30 2023-10-18T18:03:26.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd200bf550 0x7efd20153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:26.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd200bf550 0x7efd20153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.72:46334/0 (socket says 172.21.15.72:46334) 2023-10-18T18:03:26.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1f7fe700 1 -- 172.21.15.72:0/3173072238 learned_addr learned my addr 172.21.15.72:0/3173072238 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:26.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1effd700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200c2f30 0x7efd201551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:26.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1ffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd20151680 0x7efd2015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:26.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1f7fe700 1 -- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200c2f30 msgr2=0x7efd201551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:26.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1f7fe700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200c2f30 0x7efd201551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:26.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1f7fe700 1 -- 172.21.15.72:0/3173072238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd20151680 msgr2=0x7efd2015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:26.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1f7fe700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd20151680 0x7efd2015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:26.683 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1f7fe700 1 -- 172.21.15.72:0/3173072238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18009a20 con 0x7efd200bf550 2023-10-18T18:03:26.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd1f7fe700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd200bf550 0x7efd20153370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7efd10009e00 tx=0x7efd10017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:26.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd077fe700 1 -- 172.21.15.72:0/3173072238 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1000f890 con 0x7efd200bf550 2023-10-18T18:03:26.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd077fe700 1 -- 172.21.15.72:0/3173072238 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1000fc50 con 0x7efd200bf550 2023-10-18T18:03:26.684 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd100092a0 con 0x7efd200bf550 2023-10-18T18:03:26.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd077fe700 1 -- 172.21.15.72:0/3173072238 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1001b410 con 0x7efd200bf550 2023-10-18T18:03:26.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.675+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd20156090 con 0x7efd200bf550 2023-10-18T18:03:26.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.679+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd20046f00 con 0x7efd200bf550 2023-10-18T18:03:26.685 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.679+0000 7efd077fe700 1 -- 172.21.15.72:0/3173072238 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7efd1001b7c0 con 0x7efd200bf550 2023-10-18T18:03:26.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.683+0000 7efd077fe700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efd0c05a510 0x7efd0c05c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:26.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.683+0000 7efd077fe700 1 -- 172.21.15.72:0/3173072238 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7efd100762b0 con 0x7efd200bf550 2023-10-18T18:03:26.686 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.683+0000 7efd1effd700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efd0c05a510 0x7efd0c05c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:26.687 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.683+0000 7efd1effd700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efd0c05a510 0x7efd0c05c9d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7efd200b9e50 tx=0x7efd14007dd0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:26.694 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.691+0000 7efd077fe700 1 -- 172.21.15.72:0/3173072238 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7efd10049860 con 0x7efd200bf550 2023-10-18T18:03:26.783 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~296s 2023-10-18T18:03:26.784 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~241s 2023-10-18T18:03:26.784 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~196s 2023-10-18T18:03:26.784 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~222s 2023-10-18T18:03:26.784 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~208s 2023-10-18T18:03:26.784 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~255s 2023-10-18T18:03:26.889 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.883+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efd200c3bb0 con 0x7efd200bf550 2023-10-18T18:03:26.890 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.887+0000 7efd077fe700 1 -- 172.21.15.72:0/3173072238 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7efd1004ce80 con 0x7efd200bf550 2023-10-18T18:03:26.890 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:26.894 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efd0c05a510 msgr2=0x7efd0c05c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:26.894 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efd0c05a510 0x7efd0c05c9d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7efd200b9e50 tx=0x7efd14007dd0).stop 2023-10-18T18:03:26.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd200bf550 msgr2=0x7efd20153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:26.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd200bf550 0x7efd20153370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7efd10009e00 tx=0x7efd10017000).stop 2023-10-18T18:03:26.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 shutdown_connections 2023-10-18T18:03:26.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7efd0c05a510 0x7efd0c05c9d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:26.895 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd20151680 0x7efd2015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:26.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7efd200c2f30 0x7efd201551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:26.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 --2- 172.21.15.72:0/3173072238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd200bf550 0x7efd20153370 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:26.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 >> 172.21.15.72:0/3173072238 conn(0x7efd20069d80 msgr2=0x7efd200c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:26.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 shutdown_connections 2023-10-18T18:03:26.896 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:26.891+0000 7efd24907700 1 -- 172.21.15.72:0/3173072238 wait complete. 2023-10-18T18:03:26.915 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:27.060 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:03:27.068 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:03:29.917 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:30.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.075+0000 7f13f3547700 1 Processor -- start 2023-10-18T18:03:30.080 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.075+0000 7f13f3547700 1 -- start start 2023-10-18T18:03:30.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f3547700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f13ec0c2bf0 0x7f13ec065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:30.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f3547700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0be2e0 con 0x7f13ec0bd1c0 2023-10-18T18:03:30.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f3547700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0be770 con 0x7f13ec0658a0 2023-10-18T18:03:30.082 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f3547700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec0bda80 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.083 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f2545700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f13ec0658a0 0x7f13ec0630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:56884/0 (socket says 172.21.15.72:56884) 2023-10-18T18:03:30.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f2545700 1 -- 172.21.15.72:0/3765561320 learned_addr learned my addr 172.21.15.72:0/3765561320 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:03:30.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1d44700 1 --2- 172.21.15.72:0/3765561320 >> v2:172.21.15.72:3300/0 conn(0x7f13ec0c2bf0 0x7f13ec065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:30.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1543700 1 -- 172.21.15.72:0/3765561320 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754315738 0 0) 0x7f13ec0be770 con 0x7f13ec0658a0 2023-10-18T18:03:30.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1543700 1 -- 172.21.15.72:0/3765561320 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d80033f0 con 0x7f13ec0658a0 2023-10-18T18:03:30.084 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1543700 1 -- 172.21.15.72:0/3765561320 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221175974 0 0) 0x7f13ec0be2e0 con 0x7f13ec0bd1c0 2023-10-18T18:03:30.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1543700 1 -- 172.21.15.72:0/3765561320 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13ec0be770 con 0x7f13ec0bd1c0 2023-10-18T18:03:30.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1d44700 1 -- 172.21.15.72:0/3765561320 >> v1:172.21.15.179:6789/0 conn(0x7f13ec0658a0 legacy=0x7f13ec0630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:30.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1d44700 1 -- 172.21.15.72:0/3765561320 >> v1:172.21.15.72:6789/0 conn(0x7f13ec0bd1c0 legacy=0x7f13ec14ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:30.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1d44700 1 -- 172.21.15.72:0/3765561320 --> v2:172.21.15.72:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0b9580 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1d44700 1 --2- 172.21.15.72:0/3765561320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0c2bf0 0x7f13ec065c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f13dc009760 tx=0x7f13dc009060).ready entity=mon.0 client_cookie=1056d968aa5ba41e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:30.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1543700 1 -- 172.21.15.72:0/3765561320 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00d4a0 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1543700 1 -- 172.21.15.72:0/3765561320 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc00d600 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.079+0000 7f13f1543700 1 -- 172.21.15.72:0/3765561320 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00d980 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 -- 172.21.15.72:0/3765561320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0c2bf0 msgr2=0x7f13ec065c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:30.087 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 --2- 172.21.15.72:0/3765561320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0c2bf0 0x7f13ec065c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f13dc009760 tx=0x7f13dc009060).stop 2023-10-18T18:03:30.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 -- 172.21.15.72:0/3765561320 shutdown_connections 2023-10-18T18:03:30.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 --2- 172.21.15.72:0/3765561320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0c2bf0 0x7f13ec065c80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:30.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 -- 172.21.15.72:0/3765561320 >> 172.21.15.72:0/3765561320 conn(0x7f13ec062ba0 msgr2=0x7f13ec0c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:30.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 -- 172.21.15.72:0/3765561320 shutdown_connections 2023-10-18T18:03:30.088 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 -- 172.21.15.72:0/3765561320 wait complete. 2023-10-18T18:03:30.090 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 Processor -- start 2023-10-18T18:03:30.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 -- start start 2023-10-18T18:03:30.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0658a0 0x7f13ec14bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:30.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13ec0bd1c0 0x7f13ec0bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:30.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f13ec0c2bf0 0x7f13ec163920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:30.091 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0a2960 con 0x7f13ec0658a0 2023-10-18T18:03:30.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec069d50 con 0x7f13ec0bd1c0 2023-10-18T18:03:30.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.083+0000 7f13f3547700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0a16d0 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2545700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0658a0 0x7f13ec14bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:30.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f1d44700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13ec0bd1c0 0x7f13ec0bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:30.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2d46700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f13ec0c2bf0 0x7f13ec163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:30.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2545700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0658a0 0x7f13ec14bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55908/0 (socket says 172.21.15.72:55908) 2023-10-18T18:03:30.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2545700 1 -- 172.21.15.72:0/3061977841 learned_addr learned my addr 172.21.15.72:0/3061977841 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:30.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2d46700 1 -- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13ec0bd1c0 msgr2=0x7f13ec0bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:30.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2d46700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13ec0bd1c0 0x7f13ec0bfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:30.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2d46700 1 -- 172.21.15.72:0/3061977841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0658a0 msgr2=0x7f13ec14bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:30.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2d46700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0658a0 0x7f13ec14bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:30.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2d46700 1 -- 172.21.15.72:0/3061977841 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc0092a0 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2545700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0658a0 0x7f13ec14bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-10-18T18:03:30.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f2d46700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f13ec0c2bf0 0x7f13ec163920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f13e400c7c0 tx=0x7f13e4007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:30.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13e27fc700 1 -- 172.21.15.72:0/3061977841 <== mon.2 v2:172.21.15.179:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e400e4a0 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13e27fc700 1 -- 172.21.15.72:0/3061977841 <== mon.2 v2:172.21.15.179:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e400e600 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec0a0e30 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13e27fc700 1 -- 172.21.15.72:0/3061977841 <== mon.2 v2:172.21.15.179:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e400e980 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec068500 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.087+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec0a1880 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.091+0000 7f13e27fc700 1 -- 172.21.15.72:0/3061977841 <== mon.2 v2:172.21.15.179:3301/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f13e400ed30 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.091+0000 7f13e27fc700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f13d005a4e0 0x7f13d005c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:30.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.091+0000 7f13e27fc700 1 -- 172.21.15.72:0/3061977841 <== mon.2 v2:172.21.15.179:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f13e4073f20 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.091+0000 7f13f2545700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f13d005a4e0 0x7f13d005c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:30.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.091+0000 7f13f2545700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f13d005a4e0 0x7f13d005c9a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f13e8005750 tx=0x7f13e8009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:30.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.103+0000 7f13e27fc700 1 -- 172.21.15.72:0/3061977841 <== mon.2 v2:172.21.15.179:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f13ec0a1880 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.308 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.303+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f13ec0a1880 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.309 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13e27fc700 1 -- 172.21.15.72:0/3061977841 <== mon.2 v2:172.21.15.179:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f13ec0a1880 con 0x7f13ec0c2bf0 2023-10-18T18:03:30.309 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:30.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f13d005a4e0 msgr2=0x7f13d005c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:30.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f13d005a4e0 0x7f13d005c9a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f13e8005750 tx=0x7f13e8009040).stop 2023-10-18T18:03:30.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f13ec0c2bf0 msgr2=0x7f13ec163920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:30.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f13ec0c2bf0 0x7f13ec163920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f13e400c7c0 tx=0x7f13e4007ef0).stop 2023-10-18T18:03:30.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 shutdown_connections 2023-10-18T18:03:30.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f13d005a4e0 0x7f13d005c9a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:30.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f13ec0c2bf0 0x7f13ec163920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:30.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13ec0bd1c0 0x7f13ec0bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:30.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 --2- 172.21.15.72:0/3061977841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13ec0658a0 0x7f13ec14bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:30.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 >> 172.21.15.72:0/3061977841 conn(0x7f13ec062ba0 msgr2=0x7f13ec0b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:30.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 shutdown_connections 2023-10-18T18:03:30.315 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:30.307+0000 7f13f3547700 1 -- 172.21.15.72:0/3061977841 wait complete. 2023-10-18T18:03:30.331 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:33.094 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~302s 2023-10-18T18:03:33.094 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~248s 2023-10-18T18:03:33.094 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~202s 2023-10-18T18:03:33.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~228s 2023-10-18T18:03:33.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~215s 2023-10-18T18:03:33.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~261s 2023-10-18T18:03:33.095 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-10-18T18:03:33.332 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:33.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.487+0000 7f947d05b700 1 Processor -- start 2023-10-18T18:03:33.490 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.487+0000 7f947d05b700 1 -- start start 2023-10-18T18:03:33.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.487+0000 7f947d05b700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f947814cfa0 0x7f947814d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:33.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947d05b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9478065cf0 con 0x7f9478067ff0 2023-10-18T18:03:33.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947d05b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9478150350 con 0x7f94780666b0 2023-10-18T18:03:33.493 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947d05b700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780650e0 con 0x7f947814cfa0 2023-10-18T18:03:33.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947659c700 1 --1- >> v1:172.21.15.179:6789/0 conn(0x7f94780666b0 0x7f94780c1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.179:6789/0 says I am v1:172.21.15.72:56892/0 (socket says 172.21.15.72:56892) 2023-10-18T18:03:33.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947659c700 1 -- 172.21.15.72:0/4168098748 learned_addr learned my addr 172.21.15.72:0/4168098748 (peer_addr_for_me v1:172.21.15.72:0/0) 2023-10-18T18:03:33.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947759e700 1 --2- 172.21.15.72:0/4168098748 >> v2:172.21.15.179:3300/0 conn(0x7f947814cfa0 0x7f947814d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:33.495 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f9475d9b700 1 -- 172.21.15.72:0/4168098748 <== mon.1 v1:172.21.15.179:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019304771 0 0) 0x7f9478150350 con 0x7f94780666b0 2023-10-18T18:03:33.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f9475d9b700 1 -- 172.21.15.72:0/4168098748 --> v1:172.21.15.179:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94600033f0 con 0x7f94780666b0 2023-10-18T18:03:33.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f9475d9b700 1 -- 172.21.15.72:0/4168098748 <== mon.0 v1:172.21.15.72:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218691870 0 0) 0x7f9478065cf0 con 0x7f9478067ff0 2023-10-18T18:03:33.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f9475d9b700 1 -- 172.21.15.72:0/4168098748 --> v1:172.21.15.72:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9478150350 con 0x7f9478067ff0 2023-10-18T18:03:33.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947759e700 1 -- 172.21.15.72:0/4168098748 >> v1:172.21.15.179:6789/0 conn(0x7f94780666b0 legacy=0x7f94780c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:33.496 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947759e700 1 -- 172.21.15.72:0/4168098748 >> v1:172.21.15.72:6789/0 conn(0x7f9478067ff0 legacy=0x7f94780683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:33.497 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947759e700 1 -- 172.21.15.72:0/4168098748 --> v2:172.21.15.179:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780653c0 con 0x7f947814cfa0 2023-10-18T18:03:33.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f947759e700 1 --2- 172.21.15.72:0/4168098748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947814cfa0 0x7f947814d430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f9470009760 tx=0x7f9470009060).ready entity=mon.1 client_cookie=81ac7cad3faae190 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:33.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f9475d9b700 1 -- 172.21.15.72:0/4168098748 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000f510 con 0x7f947814cfa0 2023-10-18T18:03:33.498 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f9475d9b700 1 -- 172.21.15.72:0/4168098748 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947000f670 con 0x7f947814cfa0 2023-10-18T18:03:33.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.491+0000 7f9475d9b700 1 -- 172.21.15.72:0/4168098748 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000f9f0 con 0x7f947814cfa0 2023-10-18T18:03:33.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.495+0000 7f947d05b700 1 -- 172.21.15.72:0/4168098748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947814cfa0 msgr2=0x7f947814d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:33.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.495+0000 7f947d05b700 1 --2- 172.21.15.72:0/4168098748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947814cfa0 0x7f947814d430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f9470009760 tx=0x7f9470009060).stop 2023-10-18T18:03:33.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.495+0000 7f947d05b700 1 -- 172.21.15.72:0/4168098748 shutdown_connections 2023-10-18T18:03:33.499 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.495+0000 7f947d05b700 1 --2- 172.21.15.72:0/4168098748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f947814cfa0 0x7f947814d430 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:33.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.495+0000 7f947d05b700 1 -- 172.21.15.72:0/4168098748 >> 172.21.15.72:0/4168098748 conn(0x7f94780620d0 msgr2=0x7f94780624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:33.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.495+0000 7f947d05b700 1 -- 172.21.15.72:0/4168098748 shutdown_connections 2023-10-18T18:03:33.500 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.495+0000 7f947d05b700 1 -- 172.21.15.72:0/4168098748 wait complete. 2023-10-18T18:03:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.495+0000 7f947d05b700 1 Processor -- start 2023-10-18T18:03:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.495+0000 7f947d05b700 1 -- start start 2023-10-18T18:03:33.501 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947d05b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94780666b0 0x7f9478149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947d05b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9478067ff0 0x7f94781610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947d05b700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947814cfa0 0x7f94781631d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947d05b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94781537a0 con 0x7f94780666b0 2023-10-18T18:03:33.502 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947d05b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947814b980 con 0x7f9478067ff0 2023-10-18T18:03:33.503 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947d05b700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9478153c30 con 0x7f947814cfa0 2023-10-18T18:03:33.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f9476d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94780666b0 0x7f9478149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:33.504 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947659c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9478067ff0 0x7f94781610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f9476d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94780666b0 0x7f9478149d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:55918/0 (socket says 172.21.15.72:55918) 2023-10-18T18:03:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f9476d9d700 1 -- 172.21.15.72:0/2435283731 learned_addr learned my addr 172.21.15.72:0/2435283731 (peer_addr_for_me v2:172.21.15.72:0/0) 2023-10-18T18:03:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947759e700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947814cfa0 0x7f94781631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f9476d9d700 1 -- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947814cfa0 msgr2=0x7f94781631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f9476d9d700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947814cfa0 0x7f94781631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:33.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f9476d9d700 1 -- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9478067ff0 msgr2=0x7f94781610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:33.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f9476d9d700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9478067ff0 0x7f94781610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:33.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f9476d9d700 1 -- 172.21.15.72:0/2435283731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700092a0 con 0x7f94780666b0 2023-10-18T18:03:33.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f9476d9d700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94780666b0 0x7f9478149d20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f946800c490 tx=0x7f9468007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:33.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f945effd700 1 -- 172.21.15.72:0/2435283731 <== mon.0 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946800ce40 con 0x7f94780666b0 2023-10-18T18:03:33.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f945effd700 1 -- 172.21.15.72:0/2435283731 <== mon.0 v2:172.21.15.72:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946800f470 con 0x7f94780666b0 2023-10-18T18:03:33.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f945effd700 1 -- 172.21.15.72:0/2435283731 <== mon.0 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946800f830 con 0x7f94780666b0 2023-10-18T18:03:33.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94780c18f0 con 0x7f94780666b0 2023-10-18T18:03:33.507 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9478152e70 con 0x7f94780666b0 2023-10-18T18:03:33.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.499+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947806c190 con 0x7f94780666b0 2023-10-18T18:03:33.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.503+0000 7f945effd700 1 -- 172.21.15.72:0/2435283731 <== mon.0 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 4) v1 ==== 72060+0+0 (secure 0 0 0) 0x7f946800fbe0 con 0x7f94780666b0 2023-10-18T18:03:33.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.503+0000 7f945effd700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f946405a510 0x7f946405c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:33.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.503+0000 7f945effd700 1 -- 172.21.15.72:0/2435283731 <== mon.0 v2:172.21.15.72:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6580+0+0 (secure 0 0 0) 0x7f9468073eb0 con 0x7f94780666b0 2023-10-18T18:03:33.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.503+0000 7f947659c700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f946405a510 0x7f946405c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-10-18T18:03:33.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.503+0000 7f947659c700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f946405a510 0x7f946405c9d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f9478154510 tx=0x7f946c009040).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T18:03:33.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.511+0000 7f945effd700 1 -- 172.21.15.72:0/2435283731 <== mon.0 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162466 (secure 0 0 0) 0x7f9468047400 con 0x7f94780666b0 2023-10-18T18:03:33.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.707+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f94780656a0 con 0x7f94780666b0 2023-10-18T18:03:33.711 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.707+0000 7f945effd700 1 -- 172.21.15.72:0/2435283731 <== mon.0 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+195 (secure 0 0 0) 0x7f946804aa20 con 0x7f94780666b0 2023-10-18T18:03:33.711 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-10-18T18:03:33.713 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f946405a510 msgr2=0x7f946405c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:33.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f946405a510 0x7f946405c9d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f9478154510 tx=0x7f946c009040).stop 2023-10-18T18:03:33.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94780666b0 msgr2=0x7f9478149d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T18:03:33.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94780666b0 0x7f9478149d20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f946800c490 tx=0x7f9468007c90).stop 2023-10-18T18:03:33.714 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 shutdown_connections 2023-10-18T18:03:33.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:6832/33696,v1:172.21.15.179:6833/33696] conn(0x7f946405a510 0x7f946405c9d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:33.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f947814cfa0 0x7f94781631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:33.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9478067ff0 0x7f94781610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:33.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 --2- 172.21.15.72:0/2435283731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f94780666b0 0x7f9478149d20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-10-18T18:03:33.715 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 >> 172.21.15.72:0/2435283731 conn(0x7f94780620d0 msgr2=0x7f947814c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T18:03:33.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 shutdown_connections 2023-10-18T18:03:33.716 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:33.711+0000 7f947d05b700 1 -- 172.21.15.72:0/2435283731 wait complete. 2023-10-18T18:03:33.736 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2023-10-18T18:03:34.404 INFO:tasks.ceph.osd.0:Sent signal 15 2023-10-18T18:03:34.404 INFO:tasks.ceph.osd.1:Sent signal 15 2023-10-18T18:03:34.405 INFO:tasks.ceph.osd.2:Sent signal 15 2023-10-18T18:03:34.405 INFO:tasks.ceph.osd.3:Sent signal 15 2023-10-18T18:03:34.405 INFO:tasks.ceph.osd.4:Sent signal 15 2023-10-18T18:03:34.406 INFO:tasks.ceph.osd.5:Sent signal 15 2023-10-18T18:03:34.406 INFO:tasks.ceph.osd.6:Sent signal 15 2023-10-18T18:03:34.406 INFO:tasks.ceph.osd.7:Sent signal 15 2023-10-18T18:03:34.407 INFO:tasks.ceph.mon.a:Sent signal 15 2023-10-18T18:03:34.407 INFO:tasks.ceph.mon.b:Sent signal 15 2023-10-18T18:03:34.408 INFO:tasks.ceph.mon.c:Sent signal 15 2023-10-18T18:03:34.408 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-10-18T18:03:34.408 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-10-18T18:03:34.409 INFO:tasks.ceph.osd.4.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fae3aeb4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 34026) UID: 0 2023-10-18T18:03:34.409 INFO:tasks.ceph.osd.4.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fae3aeb4700 -1 osd.4 28 *** Got signal Terminated *** 2023-10-18T18:03:34.410 INFO:tasks.ceph.osd.4.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fae3aeb4700 -1 osd.4 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T18:03:34.410 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f1983680700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33218) UID: 0 2023-10-18T18:03:34.410 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f1983680700 -1 osd.0 28 *** Got signal Terminated *** 2023-10-18T18:03:34.411 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f1983680700 -1 osd.0 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T18:03:34.411 INFO:tasks.ceph.osd.5.smithi179.stderr:2023-10-18T18:03:34.402+0000 7f6c1a0f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 34029) UID: 0 2023-10-18T18:03:34.411 INFO:tasks.ceph.osd.5.smithi179.stderr:2023-10-18T18:03:34.402+0000 7f6c1a0f3700 -1 osd.5 28 *** Got signal Terminated *** 2023-10-18T18:03:34.412 INFO:tasks.ceph.osd.5.smithi179.stderr:2023-10-18T18:03:34.402+0000 7f6c1a0f3700 -1 osd.5 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T18:03:34.412 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-10-18T18:03:34.403+0000 7fbcc1858700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33216) UID: 0 2023-10-18T18:03:34.412 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-10-18T18:03:34.403+0000 7fbcc1858700 -1 osd.1 28 *** Got signal Terminated *** 2023-10-18T18:03:34.412 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-10-18T18:03:34.403+0000 7fbcc1858700 -1 osd.1 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T18:03:34.413 INFO:tasks.ceph.osd.6.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fd737e71700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 34032) UID: 0 2023-10-18T18:03:34.413 INFO:tasks.ceph.osd.6.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fd737e71700 -1 osd.6 28 *** Got signal Terminated *** 2023-10-18T18:03:34.413 INFO:tasks.ceph.osd.6.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fd737e71700 -1 osd.6 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T18:03:34.414 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f7980c3f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33219) UID: 0 2023-10-18T18:03:34.414 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f7980c3f700 -1 osd.2 28 *** Got signal Terminated *** 2023-10-18T18:03:34.414 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f7980c3f700 -1 osd.2 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T18:03:34.414 INFO:tasks.ceph.osd.7.smithi179.stderr:2023-10-18T18:03:34.402+0000 7f123e2c6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 34028) UID: 0 2023-10-18T18:03:34.415 INFO:tasks.ceph.osd.7.smithi179.stderr:2023-10-18T18:03:34.402+0000 7f123e2c6700 -1 osd.7 28 *** Got signal Terminated *** 2023-10-18T18:03:34.415 INFO:tasks.ceph.osd.7.smithi179.stderr:2023-10-18T18:03:34.402+0000 7f123e2c6700 -1 osd.7 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T18:03:34.415 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f2f10f0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33223) UID: 0 2023-10-18T18:03:34.415 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f2f10f0f700 -1 osd.3 28 *** Got signal Terminated *** 2023-10-18T18:03:34.416 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f2f10f0f700 -1 osd.3 28 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T18:03:34.416 INFO:tasks.ceph.mon.b.smithi179.stderr:2023-10-18T18:03:34.402+0000 7f88945a8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 33684) UID: 0 2023-10-18T18:03:34.416 INFO:tasks.ceph.mon.b.smithi179.stderr:2023-10-18T18:03:34.402+0000 7f88945a8700 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-10-18T18:03:34.417 INFO:tasks.ceph.mon.a.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f9cc744a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 33098) UID: 0 2023-10-18T18:03:34.417 INFO:tasks.ceph.mon.a.smithi072.stderr:2023-10-18T18:03:34.403+0000 7f9cc744a700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-18T18:03:34.417 INFO:tasks.ceph.mon.c.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fe3b4702700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 33687) UID: 0 2023-10-18T18:03:34.417 INFO:tasks.ceph.mon.c.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fe3b4702700 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-10-18T18:03:34.418 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fca4bf37700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 33689) UID: 0 2023-10-18T18:03:34.418 INFO:tasks.ceph.mgr.x.smithi179.stderr:2023-10-18T18:03:34.402+0000 7fca4bf37700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-18T18:03:34.610 INFO:tasks.ceph.mgr.y.smithi072.stderr:daemon-helper: command crashed with signal 15 2023-10-18T18:03:36.736 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T18:03:36.901 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.899+0000 7f98a369b700 1 Processor -- start 2023-10-18T18:03:36.902 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.899+0000 7f98a369b700 1 -- start start 2023-10-18T18:03:36.904 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.899+0000 7f98a369b700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 0x7f989c150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:36.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.899+0000 7f98a369b700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f989c0beac0 2023-10-18T18:03:36.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.899+0000 7f98a369b700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcaf0 con 0x7f989c067e50 2023-10-18T18:03:36.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.899+0000 7f98a369b700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c151080 con 0x7f989c0beea0 2023-10-18T18:03:36.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.899+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 msgr2=0x7f989c150b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:36.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.899+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 0x7f989c150b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:36.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.899+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f989c0beac0 legacy=0x7f989c066150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:36.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:36.903+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f989c14cba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:37.103+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 msgr2=0x7f989c150b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:37.105 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:37.103+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 0x7f989c150b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:37.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:37.107+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f989c0beac0 legacy=0x7f989c066150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:37.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:37.107+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f989c14cba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:37.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:37.507+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 msgr2=0x7f989c150b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:37.509 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:37.507+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 0x7f989c150b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:37.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:37.511+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f989c0beac0 legacy=0x7f989c066150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:37.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:37.511+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f989c14cba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:38.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:38.311+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 msgr2=0x7f989c150b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:38.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:38.311+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 0x7f989c150b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:38.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:38.315+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f989c0beac0 legacy=0x7f989c066150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:38.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:38.315+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f989c14cba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:39.905 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.899+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 msgr2=0x7f989c150b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:39.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.899+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f989c0beea0 0x7f989c150b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:39.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.899+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f989c14cba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:39.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.899+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f989c0beac0 legacy=0x7f989c066150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:39.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.899+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 0x7f98800059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:39.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.903+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 0x7f98800084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:39.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.903+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880003570 2023-10-18T18:03:39.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.903+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f9880006020 2023-10-18T18:03:39.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.903+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 msgr2=0x7f98800059f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:39.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.903+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 0x7f98800059f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:39.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.903+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 msgr2=0x7f98800084e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:39.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:39.903+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 0x7f98800084e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:40.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:40.107+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 msgr2=0x7f98800084e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:40.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:40.107+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 msgr2=0x7f98800059f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:40.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:40.107+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 0x7f98800084e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:40.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:40.107+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 0x7f98800059f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:40.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:40.511+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 msgr2=0x7f98800084e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:40.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:40.511+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 msgr2=0x7f98800059f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:40.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:40.511+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 0x7f98800084e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:40.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:40.511+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 0x7f98800059f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:41.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:41.315+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 msgr2=0x7f98800059f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:41.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:41.315+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 msgr2=0x7f98800084e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:41.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:41.315+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 0x7f98800059f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:41.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:41.315+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 0x7f98800084e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:42.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 msgr2=0x7f98800084e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:42.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880006020 0x7f98800084e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:42.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 msgr2=0x7f98800059f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:42.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880003570 0x7f98800059f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:42.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 0x7f9880010e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:42.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:42.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:03:42.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f989c067e50 2023-10-18T18:03:42.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f98800064d0 2023-10-18T18:03:42.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000e980 2023-10-18T18:03:42.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 msgr2=0x7f9880010e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:42.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 0x7f9880010e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:42.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:42.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:42.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:42.903+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:43.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.107+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 msgr2=0x7f9880010e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:43.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.107+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:43.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.107+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 0x7f9880010e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:43.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.107+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:43.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.107+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:43.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.511+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.511+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.511+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 msgr2=0x7f9880010e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.511+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:43.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:43.511+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 0x7f9880010e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:44.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:44.315+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 msgr2=0x7f9880010e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:44.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:44.315+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:44.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:44.315+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:44.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:44.315+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 0x7f9880010e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:44.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:44.315+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:45.906 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 msgr2=0x7f9880010e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:45.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000e980 0x7f9880010e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:45.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:45.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:45.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:45.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:45.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f9880003aa0 2023-10-18T18:03:45.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880004680 2023-10-18T18:03:45.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880003aa0 legacy=0x7f9880003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:45.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:45.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:45.903+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:46.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:46.107+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:46.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:46.107+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880003aa0 legacy=0x7f9880003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:46.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:46.107+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:46.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:46.511+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:46.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:46.511+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880003aa0 legacy=0x7f9880003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:46.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:46.511+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:47.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:47.315+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:47.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:47.315+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880003aa0 legacy=0x7f9880003eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:47.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:47.315+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:48.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:48.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:48.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880003aa0 legacy=0x7f9880003eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:48.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:48.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:03:48.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:48.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f989c067e50 2023-10-18T18:03:48.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880004680 2023-10-18T18:03:48.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f98800064d0 2023-10-18T18:03:48.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:48.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:48.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:48.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:48.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:48.903+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:49.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.107+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:49.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.107+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:49.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.107+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:49.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.107+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:49.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.107+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:49.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.511+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:49.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.511+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:49.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.511+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:49.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.511+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:49.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:49.511+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:50.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:50.315+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:50.315+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:50.315+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:50.315+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:50.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:50.315+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:51.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880011300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:51.907 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880011300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:51.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 msgr2=0x7f98800158b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:51.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880004680 0x7f98800158b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:51.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f989c067e50 legacy=0x7f988000d240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:51.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 0x7f9880004b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:51.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f9880004ff0 2023-10-18T18:03:51.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880003eb0 2023-10-18T18:03:51.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:51.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:51.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:51.903+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:52.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:52.107+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:52.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:52.107+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:52.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:52.107+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:52.511+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:52.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:52.511+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:52.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:52.511+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:53.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:53.315+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:53.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:53.315+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:53.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:53.315+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:54.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:54.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880003eb0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:54.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:54.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 0x7f9880018dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:54.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:03:54.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:54.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f9880004ff0 2023-10-18T18:03:54.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f98800064d0 2023-10-18T18:03:54.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f9880010e10 2023-10-18T18:03:54.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 msgr2=0x7f9880018dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:54.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:54.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 0x7f9880018dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:54.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:54.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:54.903+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:55.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.107+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 msgr2=0x7f9880018dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:55.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.107+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:55.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.107+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 0x7f9880018dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:55.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.107+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:55.111 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.107+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:55.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.511+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:55.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.511+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:55.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.511+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 msgr2=0x7f9880018dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:55.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.511+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:55.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:55.511+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 0x7f9880018dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:56.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:56.315+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:03:56.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:56.315+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 msgr2=0x7f9880018dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:03:56.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:56.315+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:03:56.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:56.315+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 0x7f9880018dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:56.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:56.315+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:57.135 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:03:57.143 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:03:57.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 msgr2=0x7f9880018dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:57.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f9880010e10 0x7f9880018dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:57.908 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:57.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:03:57.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:03:57.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:03:57.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f988000de60 2023-10-18T18:03:57.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880011290 2023-10-18T18:03:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:03:57.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:57.903+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:58.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:58.107+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:58.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:58.107+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:03:58.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:58.107+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:58.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:58.511+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:58.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:58.511+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:03:58.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:58.511+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:03:59.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:59.315+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:03:59.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:59.315+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:03:59.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:03:59.315+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:00.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:00.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:00.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:00.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:00.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:00.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:00.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:00.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f9880004ff0 2023-10-18T18:04:00.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880011290 2023-10-18T18:04:00.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f98800064d0 2023-10-18T18:04:00.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:00.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.902+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:00.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.906+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:00.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:00.906+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:01.109 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.106+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:01.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.106+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:01.110 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.106+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:01.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.110+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:01.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.110+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.510+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:01.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.510+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:01.514 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.510+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.514+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:01.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:01.514+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:02.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:02.314+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:02.317 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:02.314+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:02.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:02.314+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:02.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:02.318+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:02.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:02.318+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:03.909 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:03.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:03.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000d970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:03.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000d970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:03.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:03.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:03.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f988000de60 2023-10-18T18:04:03.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000d320 2023-10-18T18:04:03.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:03.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:03.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:03.906+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:04.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:04.110+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:04.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:04.110+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:04.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:04.110+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:04.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:04.514+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:04.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:04.514+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:04.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:04.514+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:05.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:05.318+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:05.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:05.318+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:05.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:05.318+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:06.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000de60 legacy=0x7f988000e270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:06.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f9880004ff0 2023-10-18T18:04:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f98800064d0 2023-10-18T18:04:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000d320 2023-10-18T18:04:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:06.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:06.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:06.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:06.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:06.906+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:07.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.110+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.110+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.110+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.110+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:07.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.110+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:07.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.514+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:07.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.514+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:07.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.514+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:07.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.514+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:07.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:07.514+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:08.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:08.318+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:08.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:08.318+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:08.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:08.318+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:08.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:08.318+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:08.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:08.318+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:09.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:09.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:09.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:09.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880004ff0 legacy=0x7f98800053d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:09.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:09.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f988000d970 2023-10-18T18:04:09.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880011290 2023-10-18T18:04:09.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:09.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:09.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:09.906+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d970 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:10.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:10.110+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:10.110+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:10.110+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d970 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:10.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:10.514+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:10.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:10.514+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:10.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:10.514+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d970 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:11.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:11.318+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:11.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:11.318+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:11.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:11.318+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d970 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:12.910 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:12.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:12.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d970 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:12.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:12.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:12.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:12.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:12.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f98800154b0 2023-10-18T18:04:12.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:12.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880011290 2023-10-18T18:04:12.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000d320 2023-10-18T18:04:12.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:12.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:12.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:12.906+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f988001a920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:13.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.110+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:13.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.110+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f988001a920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:13.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.110+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:13.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.110+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:13.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.110+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:13.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.514+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:13.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.514+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f988001a920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:13.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.514+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:13.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.514+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:13.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:13.514+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:14.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:14.318+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:14.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:14.318+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f988001a920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:14.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:14.318+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:14.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:14.318+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:14.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:14.318+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:15.911 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 msgr2=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:15.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000d320 0x7f98800148b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:15.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:15.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:15.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f988001a920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:15.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 0x7f9880004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:15.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f98800064d0 2023-10-18T18:04:15.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000dd80 2023-10-18T18:04:15.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800064d0 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:15.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:15.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:15.906+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:16.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:16.110+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800064d0 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:16.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:16.110+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:16.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:16.110+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:16.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:16.514+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800064d0 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:16.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:16.514+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:16.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:16.514+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:17.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:17.318+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800064d0 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:17.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:17.318+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:17.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:17.318+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:18.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:18.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000dd80 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:18.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800064d0 legacy=0x7f9880014da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:18.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:18.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:18.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:18.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f988000d320 2023-10-18T18:04:18.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880011290 2023-10-18T18:04:18.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f98800064d0 2023-10-18T18:04:18.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d320 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:18.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:18.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:18.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:18.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:18.906+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:19.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.110+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d320 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:19.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.110+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:19.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.110+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:19.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.110+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:19.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.110+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:19.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.514+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:19.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.514+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:19.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.514+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d320 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:19.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.514+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:19.519 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:19.514+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:20.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:20.318+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:20.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:20.318+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:20.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:20.318+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:20.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:20.318+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d320 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:20.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:20.318+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:21.912 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:21.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:21.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:21.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:21.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000d320 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:21.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f988000a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:21.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f98800056d0 2023-10-18T18:04:21.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000dd80 2023-10-18T18:04:21.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800056d0 legacy=0x7f9880009c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:21.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f988000a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:21.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:21.906+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f988000a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:22.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:22.110+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800056d0 legacy=0x7f9880009c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:22.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:22.110+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f988000a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:22.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:22.110+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f988000a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:22.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:22.514+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800056d0 legacy=0x7f9880009c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:22.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:22.514+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f988000a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:22.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:22.514+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f988000a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:23.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:23.318+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800056d0 legacy=0x7f9880009c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:23.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:23.318+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f988000a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:23.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:23.318+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f988000a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f988000a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f988000a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:24.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f98800056d0 legacy=0x7f9880009c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f988000a880 2023-10-18T18:04:24.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880011290 2023-10-18T18:04:24.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000dd80 2023-10-18T18:04:24.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:24.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:24.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:24.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.906+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:24.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:24.910+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:25.113 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.110+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:25.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.110+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:25.114 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.110+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:25.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.114+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:25.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.114+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:25.517 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.514+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:25.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.514+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:25.518 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.514+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:25.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.518+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:25.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:25.518+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:26.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:26.318+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:26.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:26.318+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:26.322 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:26.318+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:26.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:26.322+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:26.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:26.322+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:27.203 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:04:27.211 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:04:27.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:27.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:27.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:27.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:27.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f98800148b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:27.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 0x7f9880004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:27.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f98800064d0 2023-10-18T18:04:27.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f98800154b0 2023-10-18T18:04:27.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:27.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800064d0 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:27.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:27.910+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:28.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:28.114+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:28.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:28.114+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800064d0 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:28.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:28.114+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:28.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:28.518+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800064d0 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:28.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:28.518+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:28.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:28.518+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:29.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:29.322+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800064d0 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:29.322+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:29.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:29.322+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:30.913 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 msgr2=0x7f9880004ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:30.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800154b0 0x7f9880004ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:30.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800064d0 legacy=0x7f9880004fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:30.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:30.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:30.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f988000a880 2023-10-18T18:04:30.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880011290 2023-10-18T18:04:30.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000dd80 2023-10-18T18:04:30.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:30.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:30.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:30.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:30.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:30.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:30.910+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:31.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.114+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:31.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.114+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:31.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.114+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:31.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.114+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:31.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.114+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:31.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.518+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.518+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.518+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.518+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:31.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:31.518+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:32.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:32.322+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:32.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:32.322+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:32.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:32.322+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:32.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:32.322+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:32.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:32.322+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:33.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880014fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:33.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:33.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:33.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:33.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:33.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:33.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f98800154b0 2023-10-18T18:04:33.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f98800064d0 2023-10-18T18:04:33.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:33.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:33.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:33.910+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f9880004f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:34.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:34.114+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f9880004f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:34.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:34.114+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:34.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:34.114+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:34.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:34.518+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:34.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:34.518+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f9880004f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:34.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:34.518+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:35.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:35.322+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f9880004f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:35.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:35.322+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:35.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:35.322+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:36.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 msgr2=0x7f9880004a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:36.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f98800064d0 0x7f9880004a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:36.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800154b0 legacy=0x7f9880004f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:36.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f98800138a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:36.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880013d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:36.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f988000a880 2023-10-18T18:04:36.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000d3a0 2023-10-18T18:04:36.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:36.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000dd80 2023-10-18T18:04:36.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:36.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:36.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:36.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:36.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:36.910+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:37.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.114+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:37.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.114+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:37.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.114+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:37.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.114+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:37.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.114+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:37.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.518+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:37.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.518+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:37.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.518+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:37.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.518+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:37.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:37.518+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:38.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:38.322+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:38.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:38.322+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:38.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:38.322+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:38.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:38.322+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:38.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:38.322+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:39.914 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:39.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:39.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:39.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:39.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a880 legacy=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:39.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:39.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f9880004a80 2023-10-18T18:04:39.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880011290 2023-10-18T18:04:39.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:39.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:39.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:39.910+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880004a80 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:40.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:40.114+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:40.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:40.114+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:40.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:40.114+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880004a80 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:40.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:40.518+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:40.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:40.518+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:40.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:40.518+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880004a80 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:41.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:41.322+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:41.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:41.322+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:41.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:41.322+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880004a80 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:42.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 msgr2=0x7f988000e270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:42.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880011290 0x7f988000e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:42.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880004a80 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:42.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:42.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:42.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f9880009650 2023-10-18T18:04:42.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000d3a0 2023-10-18T18:04:42.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000dd80 2023-10-18T18:04:42.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:42.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:42.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:42.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:42.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:42.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:42.910+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:43.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.114+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:43.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.114+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:43.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.114+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:43.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.114+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:43.118 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.114+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:43.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.518+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:43.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.518+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:43.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.518+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:43.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.518+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:43.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:43.518+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:44.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:44.322+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:44.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:44.322+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:44.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:44.322+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:44.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:44.322+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:44.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:44.322+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:45.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:45.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:45.915 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:45.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:45.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:45.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 0x7f988001b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:45.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f9880011290 2023-10-18T18:04:45.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000e270 2023-10-18T18:04:45.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 msgr2=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:45.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880011290 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:45.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:45.910+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 0x7f988001b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:46.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:46.114+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880011290 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:46.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:46.114+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 msgr2=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:46.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:46.114+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 0x7f988001b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:46.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:46.518+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 msgr2=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:46.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:46.518+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880011290 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:46.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:46.518+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 0x7f988001b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:47.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:47.322+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880011290 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:47.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:47.322+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 msgr2=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:47.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:47.322+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 0x7f988001b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:48.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 msgr2=0x7f988001b8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:48.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000e270 0x7f988001b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:48.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f9880011290 legacy=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:48.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:48.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f988000a210 2023-10-18T18:04:48.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f9880009650 2023-10-18T18:04:48.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000dd80 2023-10-18T18:04:48.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:48.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800188d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:48.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:48.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:48.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:48.909+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:49.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:49.113+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800188d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:49.116 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:49.113+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:49.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:49.113+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:49.117 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:49.113+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:49.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:49.517+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800188d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:49.521 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:49.517+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:49.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:49.517+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:49.522 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:49.517+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:50.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:50.321+0000 7f98a1c38700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800188d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:50.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:50.321+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:50.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:50.321+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:50.326 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:50.321+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:51.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:51.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:51.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:51.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800188d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:51.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:51.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:51.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000d3a0 2023-10-18T18:04:51.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000a650 2023-10-18T18:04:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:51.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:51.913+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:52.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:52.117+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:52.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:52.117+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:52.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:52.117+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:52.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:52.117+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:52.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:52.521+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:52.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:52.521+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:52.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:52.521+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:52.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:52.521+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:53.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:53.325+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:53.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:53.325+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:53.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:53.325+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:53.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:53.325+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:54.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:54.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:54.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:54.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000d3a0 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:54.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f988001a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:54.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:04:54.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcaf0 con 0x7f988000a210 2023-10-18T18:04:54.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f9880009650 2023-10-18T18:04:54.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880017fe0 2023-10-18T18:04:54.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:54.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f988001a3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:54.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:54.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:54.913+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f988001a3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:55.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:55.117+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:55.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:55.117+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f988001a3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:55.117+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:55.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:55.117+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f988001a3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:55.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:55.521+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f988001a3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:55.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:55.521+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:55.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:55.521+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f988001a3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:55.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:55.521+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:56.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:56.325+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:04:56.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:56.325+0000 7f98a0c36700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f988001a3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:04:56.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:56.325+0000 7f98a0c36700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f988001a3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:56.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:56.325+0000 7f98a1c38700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:04:57.270 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:04:57.278 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:04:57.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f988001a3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:57.916 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f988001a3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:04:57.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:57.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:04:57.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:57.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:04:57.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000a650 2023-10-18T18:04:57.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000dd80 2023-10-18T18:04:57.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:57.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:57.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:57.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:57.913+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:04:58.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:58.117+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:58.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:58.117+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:58.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:58.117+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:58.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:58.117+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:04:58.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:58.521+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:58.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:58.521+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:58.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:58.521+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:58.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:58.521+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:04:59.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:59.325+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:04:59.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:59.325+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:04:59.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:59.325+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:04:59.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:04:59.325+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:00.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:00.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:00.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:00.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:00.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:05:00.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:00.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:00.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f988000a210 2023-10-18T18:05:00.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880017fe0 2023-10-18T18:05:00.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000dd80 2023-10-18T18:05:00.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f98a1c38700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:00.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f98a1c38700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:00.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:00.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:00.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:00.913+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:01.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.117+0000 7f98a1c38700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:01.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.117+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:01.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.117+0000 7f98a1c38700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:01.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.117+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:01.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.117+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:01.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.521+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:01.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.521+0000 7f98a1c38700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:01.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.521+0000 7f98a1c38700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:01.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.521+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:01.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:01.521+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:02.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:02.325+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:02.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:02.325+0000 7f98a1c38700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:02.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:02.325+0000 7f98a1c38700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:02.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:02.325+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:02.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:02.325+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:03.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:03.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:03.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:03.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:03.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:03.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 0x7f9880008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:03.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcaf0 con 0x7f988000a650 2023-10-18T18:05:03.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880009d60 2023-10-18T18:05:03.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000a650 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:03.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:03.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:03.913+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:04.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:04.117+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:04.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:04.117+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000a650 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:04.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:04.117+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:04.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:04.521+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000a650 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:04.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:04.521+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:04.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:04.521+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:05.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:05.325+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000a650 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:05.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:05.325+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:05.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:05.325+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:06.917 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:06.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880009d60 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:06.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f988000a650 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:06.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:05:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f988000a210 2023-10-18T18:05:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880017fe0 2023-10-18T18:05:06.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000dd80 2023-10-18T18:05:06.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:06.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:06.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:06.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:06.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:06.913+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:07.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.117+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:07.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.117+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:07.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.117+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:07.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.117+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:07.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.117+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:07.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.521+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:07.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.521+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:07.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.521+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:07.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.521+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:07.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:07.521+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:08.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:08.325+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:08.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:08.325+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:08.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:08.325+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:08.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:08.325+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:08.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:08.325+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:09.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:09.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:09.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:09.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:09.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:09.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:09.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcaf0 con 0x7f9880009650 2023-10-18T18:05:09.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000a650 2023-10-18T18:05:09.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:09.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:09.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:09.913+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:10.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:10.117+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:10.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:10.117+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:10.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:10.117+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:10.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:10.521+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:10.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:10.521+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:10.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:10.521+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:11.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:11.325+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:11.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:11.325+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:11.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:11.325+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:12.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:12.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:12.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009650 legacy=0x7f9880009a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:12.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:12.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:12.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:12.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880017fe0 2023-10-18T18:05:12.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f988000a650 2023-10-18T18:05:12.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98800043d0 con 0x7f988000dd80 2023-10-18T18:05:12.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:05:12.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:12.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:12.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:12.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:12.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:12.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:12.913+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:13.121 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.117+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:13.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.117+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:13.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.117+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:13.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.117+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:13.122 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.117+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:13.123 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.117+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:13.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.521+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:13.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.521+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:13.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.521+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:13.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.521+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:13.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.521+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:13.527 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:13.521+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:14.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:14.325+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:14.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:14.325+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:14.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:14.325+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:14.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:14.325+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:14.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:14.325+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:14.331 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:14.325+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:15.918 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 msgr2=0x7f9880009160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:15.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000dd80 0x7f9880009160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:15.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 msgr2=0x7f9880008c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:15.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f988000a650 0x7f9880008c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:15.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:15.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:15.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f988000a210 2023-10-18T18:05:15.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcaf0 con 0x7f9880009d60 2023-10-18T18:05:15.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:15.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:15.913+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009d60 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:16.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:16.117+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:16.119 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:16.117+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009d60 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:16.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:16.521+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:16.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:16.521+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009d60 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:17.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:17.325+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:17.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:17.325+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009d60 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:18.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f9880009d60 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:18.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f9880009650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:18.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:05:18.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:18.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f988000a210 2023-10-18T18:05:18.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bd9e0 con 0x7f98800183c0 2023-10-18T18:05:18.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000a650 2023-10-18T18:05:18.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:18.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:18.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:18.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:18.913+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:19.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:19.117+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:19.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:19.117+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:19.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:19.117+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:19.120 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:19.117+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:19.525 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:19.521+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:19.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:19.521+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:19.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:19.521+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:19.526 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:19.521+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:20.329 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:20.325+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:20.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:20.325+0000 7f98a1c38700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:20.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:20.325+0000 7f98a1437700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:20.330 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:20.325+0000 7f98a1c38700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:21.919 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:21.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a650 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:21.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:21.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a210 legacy=0x7f98800168c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:21.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 0x7f98800176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:21.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800138a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:21.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000dd80 2023-10-18T18:05:21.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f9880017bd0 2023-10-18T18:05:21.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:21.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:21.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:21.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:21.917+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:22.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:22.121+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:22.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:22.121+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:22.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:22.121+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:22.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:22.121+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:22.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:22.525+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:22.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:22.525+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:22.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:22.525+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:22.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:22.525+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:23.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:23.329+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:23.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:23.329+0000 7f98a0c36700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:23.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:23.329+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:23.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:23.329+0000 7f98a0c36700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:24.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800138a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:24.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800138a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:24.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:24.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f988000dd80 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:24.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880013d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:24.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:05:24.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:24.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcaf0 con 0x7f98800183c0 2023-10-18T18:05:24.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880017fe0 2023-10-18T18:05:24.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f9880017bd0 2023-10-18T18:05:24.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:24.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:24.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:24.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:24.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:24.917+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:25.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.121+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:25.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.121+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:25.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.121+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:25.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.121+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:25.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.121+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:25.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.525+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:25.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.525+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:25.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.525+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:25.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.525+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:25.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:25.525+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:26.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:26.329+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:26.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:26.329+0000 7f98a1437700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:26.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:26.329+0000 7f98a1c38700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:26.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:26.329+0000 7f98a1437700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:26.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:26.329+0000 7f98a1c38700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:27.345 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:05:27.353 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:05:27.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:27.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:27.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f9880013d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:27.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f9880013d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:27.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f9880016fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:27.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:27.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f988000a650 2023-10-18T18:05:27.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000a210 2023-10-18T18:05:27.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a650 legacy=0x7f9880016290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:27.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:27.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:27.917+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:28.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:28.121+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a650 legacy=0x7f9880016290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:28.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:28.121+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:28.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:28.121+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:28.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:28.525+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a650 legacy=0x7f9880016290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:28.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:28.525+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:28.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:28.525+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:29.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:29.329+0000 7f98a0c36700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a650 legacy=0x7f9880016290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:29.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:29.329+0000 7f98a1437700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:29.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:29.329+0000 7f98a1437700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:30.920 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f9892ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:30.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:30.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f9892ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a650 legacy=0x7f9880016290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:30.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:30.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f98a1c38700 1 -- reap_dead start 2023-10-18T18:05:30.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f98800170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:30.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f9892ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcaf0 con 0x7f98800183c0 2023-10-18T18:05:30.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f9892ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f9880017fe0 2023-10-18T18:05:30.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f9892ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988000aeb0 con 0x7f9880017bd0 2023-10-18T18:05:30.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f98a1c38700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f98800170b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:30.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f98800169a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:30.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f98a1c38700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f98800170b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:30.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:30.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:30.917+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:31.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.121+0000 7f98a1c38700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f98800170b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:31.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.121+0000 7f98a1c38700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f98800170b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:31.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.121+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f98800169a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:31.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.121+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:31.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.121+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:31.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.525+0000 7f98a1c38700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f98800170b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:31.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.525+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f98800169a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:31.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.525+0000 7f98a1c38700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f98800170b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:31.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.525+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:31.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:31.525+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:32.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:32.329+0000 7f98a1c38700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f98800170b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:32.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:32.329+0000 7f98a0c36700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f98800169a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:32.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:32.329+0000 7f98a1437700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:32.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:32.329+0000 7f98a1c38700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f98800170b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:32.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:32.329+0000 7f98a1437700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:33.921 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f9892ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 msgr2=0x7f98800176e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:33.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f9880017bd0 0x7f98800176e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:33.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f9892ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 msgr2=0x7f98800170b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:33.922 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f9892ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f9880017fe0 0x7f98800170b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:33.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f9892ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f98800183c0 legacy=0x7f98800169a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:33.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f9892ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:33.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f9892ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989c0bcdd0 con 0x7f988000a650 2023-10-18T18:05:33.923 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f9892ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880008b30 con 0x7f988000a210 2023-10-18T18:05:33.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a650 legacy=0x7f9880016290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:33.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:33.924 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:33.917+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:34.125 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:34.121+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a650 legacy=0x7f9880016290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:34.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:34.121+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:34.126 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:34.121+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:34.529 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:34.525+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:34.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:34.525+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:34.530 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:34.525+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a650 legacy=0x7f9880016290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:35.333 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:35.328+0000 7f98a0c36700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 msgr2=0x7f9880004e60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:35.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:35.328+0000 7f98a1437700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f988000a650 legacy=0x7f9880016290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:35.334 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:35.328+0000 7f98a0c36700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f988000a210 0x7f9880004e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:36.793 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-18T18:05:36.794 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-18T18:05:36.795 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T18:05:36.955 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.948+0000 7f8526307700 1 Processor -- start 2023-10-18T18:05:36.956 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.948+0000 7f8526307700 1 -- start start 2023-10-18T18:05:36.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8526307700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f85200bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:36.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8526307700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 0x7f85200689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:36.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8526307700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f85200bcaf0 2023-10-18T18:05:36.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8526307700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85200bf340 2023-10-18T18:05:36.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8526307700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f852013d890 2023-10-18T18:05:36.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f85200bf720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:36.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f85200bf720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:36.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 msgr2=0x7f85200689b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:36.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85200bcaf0 legacy=0x7f852013d180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:36.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:36.952+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 0x7f85200689b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:37.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.156+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f85200bf720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:37.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.156+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f85200bf720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.156+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 msgr2=0x7f85200689b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.156+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85200bcaf0 legacy=0x7f852013d180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:37.161 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.156+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 0x7f85200689b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:37.565 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.560+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f85200bf720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.560+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f85200bf720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.560+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 msgr2=0x7f85200689b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.560+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85200bcaf0 legacy=0x7f852013d180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:37.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:37.560+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 0x7f85200689b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:38.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:38.364+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f85200bf720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:38.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:38.364+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f85200bf720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:38.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:38.364+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85200bcaf0 legacy=0x7f852013d180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:38.367 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:38.364+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 msgr2=0x7f85200689b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:38.368 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:38.364+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 0x7f85200689b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:39.959 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 msgr2=0x7f85200689b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:39.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f852013d890 0x7f85200689b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:39.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f85200bf720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:39.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f85200bf720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:39.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85200bcaf0 legacy=0x7f852013d180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f85040060a0 2023-10-18T18:05:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504003590 2023-10-18T18:05:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f8504008560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:39.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:39.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:39.956+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:40.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:40.160+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f8504008560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:40.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:40.160+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:40.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:40.160+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:40.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:40.564+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f8504008560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:40.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:40.564+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:40.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:40.564+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:41.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:41.368+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f8504008560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:41.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:41.368+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:41.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:41.368+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:42.960 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:42.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:42.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f8504008560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:42.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:05:42.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:42.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:42.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f85040060a0 2023-10-18T18:05:42.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85200bf340 2023-10-18T18:05:42.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504003590 2023-10-18T18:05:42.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:42.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:42.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:42.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:42.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:42.956+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f850400a2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:43.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.160+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f850400a2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:43.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.160+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:43.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.160+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:43.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.160+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:43.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.160+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:43.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.564+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:43.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.564+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f850400a2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:43.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.564+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:43.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.564+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:43.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:43.564+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:44.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:44.368+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f850400a2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:44.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:44.368+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:44.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:44.368+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:44.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:44.368+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:44.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:44.368+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.952+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.952+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504003590 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:45.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.952+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:45.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.952+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:45.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.952+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040060a0 legacy=0x7f850400a2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:45.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.952+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:45.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.952+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f850400f470 2023-10-18T18:05:45.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.952+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85040085c0 2023-10-18T18:05:45.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.956+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f470 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:45.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.956+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:45.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:45.956+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:46.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:46.160+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f470 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:46.160+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:46.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:46.160+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:46.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:46.564+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f470 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:46.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:46.564+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:46.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:46.564+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:47.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:47.368+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f470 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:47.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:47.368+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:47.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:47.368+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:48.961 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:48.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:48.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f470 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:48.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:48.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:05:48.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:48.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f8504012f80 2023-10-18T18:05:48.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85200bf340 2023-10-18T18:05:48.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f85040085c0 2023-10-18T18:05:48.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:48.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:48.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:48.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f8504015370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:48.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:48.956+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:49.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.160+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:49.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.160+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f8504015370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:49.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.160+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:49.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.160+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:49.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.160+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:49.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.564+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:49.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.564+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:49.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.564+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f8504015370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:49.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.564+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:49.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:49.564+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:50.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:50.368+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:50.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:50.368+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:50.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:50.368+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:50.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:50.368+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f8504015370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:50.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:50.368+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:51.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:51.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:51.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 msgr2=0x7f8504009e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:51.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85200bf340 0x7f8504009e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:51.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f8504015370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:51.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 0x7f8504008c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:51.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f8504009180 2023-10-18T18:05:51.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400a350 2023-10-18T18:05:51.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:51.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 msgr2=0x7f8504008c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:51.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:51.956+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 0x7f8504008c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:52.162 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:52.160+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:52.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:52.160+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 msgr2=0x7f8504008c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:52.163 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:52.160+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 0x7f8504008c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:52.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:52.564+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:52.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:52.564+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 msgr2=0x7f8504008c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:52.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:52.564+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 0x7f8504008c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:53.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:53.368+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:53.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:53.368+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 msgr2=0x7f8504008c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:53.372 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:53.368+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 0x7f8504008c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:54.962 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 msgr2=0x7f8504008c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:54.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400a350 0x7f8504008c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:54.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:54.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:54.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:05:54.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f8504009180 2023-10-18T18:05:54.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f8504012f80 2023-10-18T18:05:54.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85040085c0 2023-10-18T18:05:54.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:54.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:54.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:54.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:54.956+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:55.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:55.160+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:55.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:55.160+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:55.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:55.160+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:55.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:55.160+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:55.566 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:55.564+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:55.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:55.564+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:55.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:55.564+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:55.567 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:55.564+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:56.373 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:56.368+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:05:56.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:56.368+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:05:56.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:56.368+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:56.374 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:56.368+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:05:57.410 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:05:57.418 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:05:57.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 msgr2=0x7f850400ef80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:57.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040085c0 0x7f850400ef80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:05:57.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:57.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504009180 legacy=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:05:57.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 0x7f8504005a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:57.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 0x7f8504010f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:05:57.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504012880 2023-10-18T18:05:57.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504008c90 2023-10-18T18:05:57.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:57.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 msgr2=0x7f8504010f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:57.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:57.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:57.960+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 0x7f8504010f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:05:58.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:58.164+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 msgr2=0x7f8504010f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:58.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:58.164+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:58.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:58.164+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 0x7f8504010f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:58.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:58.164+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:05:58.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:58.568+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 msgr2=0x7f8504010f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:58.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:58.568+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 0x7f8504010f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:58.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:58.568+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:58.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:58.568+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:05:59.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:59.372+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 msgr2=0x7f8504010f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:05:59.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:59.372+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:05:59.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:59.372+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 0x7f8504010f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:05:59.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:05:59.372+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:00.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:00.963 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504008c90 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:00.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 msgr2=0x7f8504010f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:00.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504012880 0x7f8504010f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:00.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:00.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:00.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:00.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f8504012f80 2023-10-18T18:06:00.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504008c90 2023-10-18T18:06:00.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504009180 2023-10-18T18:06:00.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:00.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:00.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:00.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:00.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:00.960+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:01.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.164+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:01.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.164+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:01.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.164+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:01.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.164+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:01.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.164+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:01.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.568+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:01.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.568+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:01.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.568+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:01.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.568+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:01.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:01.568+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:02.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:02.372+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:02.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:02.372+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:02.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:02.372+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:02.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:02.372+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:02.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:02.372+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:03.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:03.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:03.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:03.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:03.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012f80 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:03.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 0x7f8504010670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:03.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f85040085a0 2023-10-18T18:06:03.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504012470 2023-10-18T18:06:03.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040085a0 legacy=0x7f850400ff60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:03.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 msgr2=0x7f8504010670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:03.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:03.956+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 0x7f8504010670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:04.164 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:04.160+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040085a0 legacy=0x7f850400ff60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:04.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:04.160+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 msgr2=0x7f8504010670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:04.165 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:04.160+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 0x7f8504010670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:04.569 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:04.564+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040085a0 legacy=0x7f850400ff60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:04.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:04.564+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 msgr2=0x7f8504010670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:04.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:04.564+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 0x7f8504010670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:05.370 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:05.368+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040085a0 legacy=0x7f850400ff60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:05.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:05.368+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 msgr2=0x7f8504010670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:05.371 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:05.368+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 0x7f8504010670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:06.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 msgr2=0x7f8504010670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:06.964 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012470 0x7f8504010670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:06.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040085a0 legacy=0x7f850400ff60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:06.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:06.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:06.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:06.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f850400f850 2023-10-18T18:06:06.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504008c90 2023-10-18T18:06:06.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504009180 2023-10-18T18:06:06.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:06.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:06.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:06.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:06.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:06.960+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:07.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.164+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:07.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.164+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:07.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.164+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:07.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.164+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:07.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.164+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:07.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.568+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:07.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.568+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:07.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.568+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:07.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.568+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:07.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:07.568+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:08.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:08.372+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:08.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:08.372+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:08.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:08.372+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:08.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:08.372+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:08.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:08.372+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:09.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:09.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504009180 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:09.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:09.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:09.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:09.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:09.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f8504012350 2023-10-18T18:06:09.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400e650 2023-10-18T18:06:09.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:09.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:09.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:09.960+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:10.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:10.164+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:10.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:10.164+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:10.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:10.164+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:10.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:10.568+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:10.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:10.568+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:10.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:10.568+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:11.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:11.372+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:11.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:11.372+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:11.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:11.372+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:12.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:12.965 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:12.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:12.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:12.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:12.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f850400f850 2023-10-18T18:06:12.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f8504012350 2023-10-18T18:06:12.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400e650 2023-10-18T18:06:12.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:12.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:12.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:12.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:12.960+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:13.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:13.164+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:13.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:13.164+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:13.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:13.164+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:13.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:13.164+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:13.570 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:13.568+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:13.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:13.568+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:13.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:13.568+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:13.571 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:13.568+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:14.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:14.372+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:14.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:14.372+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:14.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:14.372+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:14.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:14.372+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:15.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 msgr2=0x7f850400ea60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:15.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400e650 0x7f850400ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:15.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504012350 legacy=0x7f850400df40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f8504011c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 0x7f8504005a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504009110 2023-10-18T18:06:15.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504008c90 2023-10-18T18:06:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:15.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:15.960+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:16.166 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:16.164+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:16.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:16.164+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:16.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:16.164+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:16.167 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:16.164+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:16.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:16.568+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:16.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:16.568+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:16.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:16.568+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:16.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:16.568+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:17.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:17.372+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:17.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:17.372+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:17.375 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:17.372+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:17.376 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:17.372+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:18.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:18.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:18.966 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 msgr2=0x7f8504005a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:18.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504009110 0x7f8504005a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:18.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:18.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f8504017f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:18.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:18.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f850400af20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:18.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f850400f850 2023-10-18T18:06:18.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400ef50 2023-10-18T18:06:18.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504008c90 2023-10-18T18:06:18.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f8504017f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:18.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f8504017f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:18.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:18.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:18.964+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:19.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.168+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f8504017f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:19.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.168+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f850400af20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:19.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.168+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f8504017f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:19.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.168+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:19.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.168+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:19.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.571+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f8504017f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:19.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.571+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f850400af20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:19.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.571+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f8504017f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:19.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.571+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:19.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:19.571+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:20.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:20.375+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f850400af20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:20.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:20.375+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f8504017f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:20.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:20.375+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f8504017f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:20.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:20.375+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:20.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:20.375+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f8504017f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:21.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f8504017f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400f850 legacy=0x7f850400af20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 0x7f8504014f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f8504015410 2023-10-18T18:06:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85040059f0 2023-10-18T18:06:21.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504015410 legacy=0x7f8504015820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:21.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504014f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:21.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:21.963+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 0x7f8504014f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:22.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:22.167+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504014f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:22.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:22.167+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504015410 legacy=0x7f8504015820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:22.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:22.167+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 0x7f8504014f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:22.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:22.571+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504014f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:22.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:22.571+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504015410 legacy=0x7f8504015820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:22.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:22.571+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 0x7f8504014f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:23.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:23.375+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504015410 legacy=0x7f8504015820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:23.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:23.375+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504014f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:23.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:23.375+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 0x7f8504014f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:24.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504014f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:24.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040059f0 0x7f8504014f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:24.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504015410 legacy=0x7f8504015820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:24.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f850400e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:24.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:24.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:24.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f850400eb40 2023-10-18T18:06:24.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400ef50 2023-10-18T18:06:24.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504008c90 2023-10-18T18:06:24.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f850400e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:24.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f850400e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:24.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400eb40 legacy=0x7f8504010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:24.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:24.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:24.963+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:25.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.167+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f850400e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:25.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.167+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f850400e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:25.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.167+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400eb40 legacy=0x7f8504010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:25.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.167+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:25.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.167+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:25.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.571+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f850400e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:25.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.571+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f850400e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:25.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.571+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400eb40 legacy=0x7f8504010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:25.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.571+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:25.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:25.571+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:26.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:26.375+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f850400e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:26.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:26.375+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400eb40 legacy=0x7f8504010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:26.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:26.375+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:26.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:26.375+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f850400e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:26.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:26.375+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:27.477 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:06:27.485 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:06:27.967 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:27.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:27.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f850400e650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:27.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f850400e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:27.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400eb40 legacy=0x7f8504010c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:27.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 0x7f85040157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:27.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f85040059f0 2023-10-18T18:06:27.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504012360 2023-10-18T18:06:27.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:27.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:27.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:27.963+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:28.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:28.167+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:28.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:28.167+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:28.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:28.167+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:28.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:28.571+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:28.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:28.571+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:28.575 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:28.571+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:29.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:29.375+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:29.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:29.375+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:29.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:29.375+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:30.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:30.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504012360 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:30.968 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:30.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:30.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:30.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:30.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f85040059f0 2023-10-18T18:06:30.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400ef50 2023-10-18T18:06:30.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504008c90 2023-10-18T18:06:30.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8525b06700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:30.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8525b06700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:30.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:30.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:30.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:30.963+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:31.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.167+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.167+0000 7f8525b06700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.167+0000 7f8525b06700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:31.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.167+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:31.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.167+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:31.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.571+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:31.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.571+0000 7f8525b06700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:31.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.571+0000 7f8525b06700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:31.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.571+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:31.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:31.571+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:32.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:32.375+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:32.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:32.375+0000 7f8525b06700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:32.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:32.375+0000 7f8525b06700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:32.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:32.375+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:32.379 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:32.375+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:33.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:33.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:33.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:33.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:33.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:33.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:33.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f8504014ee0 2023-10-18T18:06:33.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400cf40 2023-10-18T18:06:33.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:33.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:33.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:33.963+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:34.169 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:34.167+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:34.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:34.167+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:34.170 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:34.167+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:34.573 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:34.571+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:34.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:34.571+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:34.574 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:34.571+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:35.377 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:35.375+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:35.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:35.375+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:35.378 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:35.375+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:36.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:36.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:36.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:36.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:36.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:36.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:36.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f85040059f0 2023-10-18T18:06:36.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400ef50 2023-10-18T18:06:36.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504008c90 2023-10-18T18:06:36.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:36.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.963+0000 7f8525b06700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:36.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.967+0000 7f8525b06700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:36.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.967+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:36.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:36.967+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:37.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.171+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:37.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.171+0000 7f8525b06700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:37.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.171+0000 7f8525b06700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:37.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.171+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:37.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.171+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:37.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.575+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:37.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.575+0000 7f8525b06700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:37.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.575+0000 7f8525b06700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:37.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.575+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:37.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:37.575+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:38.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:38.379+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:38.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:38.379+0000 7f8525b06700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:38.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:38.379+0000 7f8525b06700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:38.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:38.379+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:38.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:38.379+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:39.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.963+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:39.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.963+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:39.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.963+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:39.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:39.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.963+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040059f0 legacy=0x7f850400f850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.963+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.963+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f8504014ee0 2023-10-18T18:06:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.967+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400cf40 2023-10-18T18:06:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.967+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.967+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:39.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:39.967+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:40.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:40.171+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:40.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:40.171+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:40.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:40.171+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:40.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:40.575+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:40.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:40.575+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:40.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:40.575+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:41.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:41.379+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:41.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:41.379+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:41.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:41.379+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:42.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:42.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:42.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f8504014ee0 legacy=0x7f8504011f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:42.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400ef50 2023-10-18T18:06:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:42.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f850400cf40 2023-10-18T18:06:42.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850400d860 con 0x7f8504008c90 2023-10-18T18:06:42.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:42.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:42.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:42.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:42.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:42.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:42.967+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:43.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.171+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:43.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.171+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:43.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.171+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:43.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.171+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:43.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.171+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:43.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.171+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:43.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.575+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:43.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.575+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:43.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.575+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:43.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.575+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:43.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.575+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:43.579 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:43.575+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:44.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:44.379+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:44.379+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:44.379+0000 7f8525b06700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:44.379+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:44.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:44.379+0000 7f8525b06700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:44.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:44.379+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:45.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:45.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:45.970 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f850400e0b0 2023-10-18T18:06:45.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f85040059f0 2023-10-18T18:06:45.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:45.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:45.967+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:46.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:46.171+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:46.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:46.171+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:46.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:46.575+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:46.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:46.575+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:47.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:47.379+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:47.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:47.379+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:48.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:48.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:48.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:48.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:48.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:48.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f85040059f0 2023-10-18T18:06:48.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400ef50 2023-10-18T18:06:48.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f850400cf40 2023-10-18T18:06:48.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:48.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:48.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:48.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:48.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:48.967+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:49.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.171+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:49.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.171+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:49.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.171+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:49.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.171+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:49.175 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.171+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:49.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.575+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:49.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.575+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:49.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.575+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:49.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.575+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:49.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:49.575+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:50.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:50.379+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:50.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:50.379+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:50.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:50.379+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:50.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:50.379+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:50.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:50.379+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:51.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400d350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:51.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400d350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:51.971 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:51.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f850400ef50 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:51.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f85040059f0 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:51.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:51.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f850400e0b0 2023-10-18T18:06:51.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504008c90 2023-10-18T18:06:51.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:51.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:51.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:51.967+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:52.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:52.171+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:52.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:52.171+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:52.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:52.171+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:52.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:52.575+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:52.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:52.575+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:52.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:52.575+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:53.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:53.379+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:53.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:53.379+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:53.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:53.379+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:54.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:54.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:54.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f850400e0b0 legacy=0x7f850400e4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:54.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:54.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:06:54.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:54.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f850400cf40 2023-10-18T18:06:54.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85040059f0 2023-10-18T18:06:54.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504008c90 2023-10-18T18:06:54.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:54.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:54.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:54.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:54.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:54.967+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:55.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.171+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:55.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.171+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:55.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.171+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:55.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.171+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:55.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.171+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:55.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.575+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.575+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.575+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.575+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:55.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:55.575+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:56.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:56.379+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:06:56.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:56.379+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:56.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:56.379+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:06:56.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:56.379+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:06:56.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:56.379+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:06:57.554 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:06:57.563 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:06:57.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f85040095d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:57.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f85040095d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:57.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:57.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:06:57.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:06:57.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 0x7f850400f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:06:57.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f8504014ea0 2023-10-18T18:06:57.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400d320 2023-10-18T18:06:57.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504014ea0 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:57.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 msgr2=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:57.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:57.967+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 0x7f850400f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:06:58.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:58.171+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504014ea0 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:58.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:58.171+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 msgr2=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:58.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:58.171+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 0x7f850400f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:06:58.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:58.575+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504014ea0 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:58.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:58.575+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 msgr2=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:58.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:58.575+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 0x7f850400f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:06:59.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:59.379+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504014ea0 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:06:59.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:59.379+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 msgr2=0x7f850400f850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:06:59.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:06:59.379+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 0x7f850400f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:00.972 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 msgr2=0x7f850400f850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:00.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400d320 0x7f850400f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:00.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504014ea0 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:00.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 0x7f8504019120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:00.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:07:00.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:00.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f850400cf40 2023-10-18T18:07:00.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85040059f0 2023-10-18T18:07:00.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504016d30 2023-10-18T18:07:00.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:00.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 msgr2=0x7f8504019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:00.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:00.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:00.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:00.967+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 0x7f8504019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:01.173 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.171+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:01.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.171+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:01.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.171+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 msgr2=0x7f8504019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:01.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.171+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:01.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.171+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 0x7f8504019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:01.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.575+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:01.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.575+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:01.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.575+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 msgr2=0x7f8504019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:01.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.575+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 0x7f8504019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:01.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:01.575+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:02.381 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:02.379+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:02.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:02.379+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:02.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:02.379+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 msgr2=0x7f8504019120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:02.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:02.379+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:02.383 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:02.379+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 0x7f8504019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:03.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 msgr2=0x7f8504019120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:03.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f8504016d30 0x7f8504019120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:03.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f85040157f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:03.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f85040157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:03.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:03.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 0x7f8504016430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:03.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f8504008c90 2023-10-18T18:07:03.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85040095d0 2023-10-18T18:07:03.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:03.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:03.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:03.966+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:04.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:04.170+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:04.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:04.170+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:04.174 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:04.170+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:04.577 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:04.574+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:04.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:04.574+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:04.578 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:04.574+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:05.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:05.378+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:05.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:05.378+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:05.382 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:05.378+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:06.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.966+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:06.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.966+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f85040095d0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:06.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.966+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:06.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.966+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:06.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.966+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:07:06.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.966+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:06.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.966+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f8504008c90 2023-10-18T18:07:06.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.966+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85040059f0 2023-10-18T18:07:06.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.966+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504016920 2023-10-18T18:07:06.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.970+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:06.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.970+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:06.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.970+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:06.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.970+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:06.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:06.970+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:07.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.174+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:07.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.174+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:07.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.174+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:07.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.174+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:07.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.174+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:07.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.578+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:07.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.578+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:07.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.578+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:07.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.578+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:07.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:07.578+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:08.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:08.382+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:08.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:08.382+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:08.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:08.382+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:08.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:08.382+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:08.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:08.382+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:09.973 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:09.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:09.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:09.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f850400cf40 2023-10-18T18:07:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85040157f0 2023-10-18T18:07:09.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:09.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:09.970+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:10.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:10.174+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:10.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:10.174+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:10.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:10.174+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:10.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:10.578+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:10.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:10.578+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:10.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:10.578+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:11.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:11.382+0000 7f8525305700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:11.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:11.382+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:11.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:11.382+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:12.974 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:12.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:12.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:12.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:12.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:07:12.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f8504008c90 2023-10-18T18:07:12.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f850400cf40 2023-10-18T18:07:12.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504016920 2023-10-18T18:07:12.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:12.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:12.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:12.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:12.970+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:13.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:13.174+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:13.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:13.174+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:13.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:13.174+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:13.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:13.174+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:13.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:13.578+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:13.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:13.578+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:13.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:13.578+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:13.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:13.578+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:14.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:14.382+0000 7f8525b06700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:14.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:14.382+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:14.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:14.382+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:14.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:14.382+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:15.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 msgr2=0x7f8504011860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:15.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504016920 0x7f8504011860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:15.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400cf40 legacy=0x7f8504011d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:15.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f8504008c90 legacy=0x7f8504015d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:15.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:15.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:15.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f85040059f0 2023-10-18T18:07:15.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f85040157f0 2023-10-18T18:07:15.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:15.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:15.970+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:16.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:16.174+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:16.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:16.174+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:16.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:16.174+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:16.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:16.174+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:16.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:16.578+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:16.578+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:16.578+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:16.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:16.578+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:17.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:17.382+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:17.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:17.382+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:17.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:17.382+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:17.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:17.382+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:18.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f85040157f0 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 msgr2=0x7f8504016430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f85040059f0 0x7f8504016430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:18.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:07:18.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:18.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:18.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:18.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504014ea0 2023-10-18T18:07:18.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f850400cf40 2023-10-18T18:07:18.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850400d860 con 0x7f8504008c90 2023-10-18T18:07:18.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:18.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:18.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8525b06700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:18.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:18.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8525b06700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:18.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:18.970+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:19.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.174+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:19.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.174+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:19.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.174+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:19.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.174+0000 7f8525b06700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:19.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.174+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:19.179 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.174+0000 7f8525b06700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:19.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.578+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:19.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.578+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:19.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.578+0000 7f8525b06700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:19.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.578+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:19.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.578+0000 7f8525b06700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:19.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:19.578+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:20.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:20.382+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:20.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:20.382+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:20.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:20.382+0000 7f8525b06700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:20.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:20.382+0000 7f8524b04700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:20.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:20.382+0000 7f8525b06700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:20.387 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:20.382+0000 7f8524b04700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:21.975 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:21.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:21.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400e380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:21.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:21.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:21.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:21.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f85040157f0 2023-10-18T18:07:21.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520068ea0 con 0x7f850400f020 2023-10-18T18:07:21.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:21.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:21.970+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:22.176 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:22.174+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:22.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:22.174+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:22.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:22.578+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:22.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:22.578+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:23.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:23.382+0000 7f8525305700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:23.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:23.382+0000 7f8524b04700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:24.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:24.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:24.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:24.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:24.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:07:24.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f850400f020 2023-10-18T18:07:24.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504014ea0 2023-10-18T18:07:24.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f8504008c90 2023-10-18T18:07:24.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:24.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:24.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:24.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:24.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:24.970+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:25.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.174+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:25.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.174+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:25.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.174+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:25.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.174+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:25.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.174+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:25.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.578+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:25.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.578+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:25.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.578+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:25.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.578+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:25.583 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:25.578+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:26.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:26.382+0000 7f8524b04700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:26.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:26.382+0000 7f8524b04700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:26.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:26.382+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:26.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:26.382+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:26.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:26.382+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:27.626 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:07:27.634 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:07:27.976 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8516ffd700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:27.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f8504012010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f85040157f0 2023-10-18T18:07:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f850400cf40 2023-10-18T18:07:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f8504012010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:27.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:27.970+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f8504012010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:28.177 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:28.174+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:28.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:28.174+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f8504012010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:28.178 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:28.174+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f8504012010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:28.581 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:28.578+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:28.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:28.578+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f8504012010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:28.582 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:28.578+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f8504012010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:29.385 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:29.382+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:29.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:29.382+0000 7f8525305700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f8504012010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:29.386 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:29.382+0000 7f8525305700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f8504012010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:30.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8516ffd700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f8504012010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:30.977 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f8504012010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:30.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:30.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:30.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8525b06700 1 -- reap_dead start 2023-10-18T18:07:30.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8516ffd700 1 -- --> v1:172.21.15.72:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8520157bf0 con 0x7f85040157f0 2023-10-18T18:07:30.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8516ffd700 1 -- --> v1:172.21.15.179:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8504009e00 con 0x7f850400f020 2023-10-18T18:07:30.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8516ffd700 1 -- --> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504008c90 2023-10-18T18:07:30.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:30.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:30.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:30.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:30.974+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:31.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:31.178+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:31.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:31.178+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:31.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:31.178+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:31.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:31.178+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:31.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:31.582+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:31.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:31.582+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:31.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:31.582+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:31.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:31.582+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:32.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:32.386+0000 7f8524b04700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.72:6789/0 2023-10-18T18:07:32.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:32.386+0000 7f8525305700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3301/0 2023-10-18T18:07:32.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:32.386+0000 7f8525b06700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.179:6789/0 2023-10-18T18:07:32.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:32.386+0000 7f8525305700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:33.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8516ffd700 1 -- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 msgr2=0x7f850400d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:33.978 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.179:3301/0,v1:172.21.15.179:6790/0] conn(0x7f8504008c90 0x7f850400d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-10-18T18:07:33.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8516ffd700 1 -- >> v1:172.21.15.179:6789/0 conn(0x7f850400f020 legacy=0x7f8504011860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:33.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8516ffd700 1 -- >> v1:172.21.15.72:6789/0 conn(0x7f85040157f0 legacy=0x7f850400e380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T18:07:33.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8516ffd700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:33.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8516ffd700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-10-18T18:07:33.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8516ffd700 1 -- --> v2:172.21.15.72:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520154220 con 0x7f8504014ea0 2023-10-18T18:07:33.980 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8516ffd700 1 -- --> v2:172.21.15.179:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8520157970 con 0x7f850400cf40 2023-10-18T18:07:33.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:33.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:33.981 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:33.982 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:33.974+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400ea90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-10-18T18:07:34.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:34.178+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:34.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:34.178+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:34.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:34.178+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:34.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:34.178+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400ea90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-10-18T18:07:34.585 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:34.582+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:34.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:34.582+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:34.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:34.582+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:34.586 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:34.582+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400ea90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-10-18T18:07:35.390 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:35.386+0000 7f8525305700 1 -- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 msgr2=0x7f850400de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.72:3300/0 2023-10-18T18:07:35.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:35.386+0000 7f8525305700 1 --2- >> v2:172.21.15.72:3300/0 conn(0x7f8504014ea0 0x7f850400de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:35.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:35.386+0000 7f8524b04700 1 -- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 msgr2=0x7f850400ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.179:3300/0 2023-10-18T18:07:35.391 INFO:teuthology.orchestra.run.smithi072.stderr:2023-10-18T18:07:35.386+0000 7f8524b04700 1 --2- >> v2:172.21.15.179:3300/0 conn(0x7f850400cf40 0x7f850400ea90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-10-18T18:07:36.841 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-18T18:07:36.842 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2023-10-18T18:07:36.843 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-18T18:07:36.843 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T18:07:36.844 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T18:07:36.845 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1451, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-10-18T18:07:36.845 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T18:07:36.846 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T18:07:36.846 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T18:07:36.847 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1451, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2023-10-18T18:07:36.848 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T18:07:36.848 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-10-18T18:07:36.849 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T18:07:36.850 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1451, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e' 2023-10-18T18:07:36.851 INFO:tasks.ceph.mds.e:Stopped 2023-10-18T18:07:36.851 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T18:07:36.851 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T18:07:36.852 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1451, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi179 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-10-18T18:07:36.853 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T18:07:36.854 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T18:07:36.854 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T18:07:36.855 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1451, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi179 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2023-10-18T18:07:36.856 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T18:07:36.856 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-10-18T18:07:36.857 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T18:07:36.858 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1451, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi179 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f' 2023-10-18T18:07:36.859 INFO:tasks.ceph.mds.f:Stopped 2023-10-18T18:07:36.859 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-18T18:07:36.860 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-18T18:07:36.860 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.861 INFO:tasks.ceph.osd.0:Stopped 2023-10-18T18:07:36.861 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-18T18:07:36.862 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.862 INFO:tasks.ceph.osd.1:Stopped 2023-10-18T18:07:36.863 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-18T18:07:36.863 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.864 INFO:tasks.ceph.osd.2:Stopped 2023-10-18T18:07:36.864 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-18T18:07:36.865 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.865 INFO:tasks.ceph.osd.3:Stopped 2023-10-18T18:07:36.866 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-18T18:07:36.866 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.867 INFO:tasks.ceph.osd.4:Stopped 2023-10-18T18:07:36.867 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-18T18:07:36.868 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.868 INFO:tasks.ceph.osd.5:Stopped 2023-10-18T18:07:36.869 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-18T18:07:36.869 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.870 INFO:tasks.ceph.osd.6:Stopped 2023-10-18T18:07:36.870 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-18T18:07:36.871 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.871 INFO:tasks.ceph.osd.7:Stopped 2023-10-18T18:07:36.872 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-18T18:07:36.872 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-18T18:07:36.873 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T18:07:36.874 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1451, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y' 2023-10-18T18:07:36.874 INFO:tasks.ceph.mgr.y:Stopped 2023-10-18T18:07:36.875 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-18T18:07:36.875 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.876 INFO:tasks.ceph.mgr.x:Stopped 2023-10-18T18:07:36.876 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-18T18:07:36.877 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-18T18:07:36.877 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.878 INFO:tasks.ceph.mon.a:Stopped 2023-10-18T18:07:36.878 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-18T18:07:36.879 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.879 INFO:tasks.ceph.mon.b:Stopped 2023-10-18T18:07:36.880 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-18T18:07:36.880 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T18:07:36.881 INFO:tasks.ceph.mon.c:Stopped 2023-10-18T18:07:36.881 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-18T18:07:36.882 DEBUG:teuthology.orchestra.run.smithi072:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'Replacing daemon mds' | head -n 1 2023-10-18T18:07:36.938 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T18:07:36.939 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-18T18:07:37.119 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T18:07:37.119 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-18T18:07:37.391 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T18:07:37.392 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-18T18:07:37.598 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T18:07:37.598 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-18T18:07:37.770 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T18:07:37.771 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-18T18:07:37.934 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T18:07:37.935 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-18T18:07:38.138 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T18:07:38.139 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-18T18:07:38.285 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T18:07:38.286 DEBUG:teuthology.orchestra.run.smithi179:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-18T18:07:38.437 INFO:tasks.ceph:Archiving mon data... 2023-10-18T18:07:38.439 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/data/mon.a.tgz 2023-10-18T18:07:38.439 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2023-10-18T18:07:38.455 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.K3VoFdHlux 2023-10-18T18:07:38.455 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.K3VoFdHlux 2023-10-18T18:07:38.719 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.K3VoFdHlux 2023-10-18T18:07:38.811 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.K3VoFdHlux is 805KB 2023-10-18T18:07:38.892 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.K3VoFdHlux 2023-10-18T18:07:38.912 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/data/mon.b.tgz 2023-10-18T18:07:38.912 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2023-10-18T18:07:38.919 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.nGO4iPLwJ5 2023-10-18T18:07:38.920 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nGO4iPLwJ5 2023-10-18T18:07:39.191 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.nGO4iPLwJ5 2023-10-18T18:07:39.211 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.nGO4iPLwJ5 is 951KB 2023-10-18T18:07:39.286 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.nGO4iPLwJ5 2023-10-18T18:07:39.293 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/data/mon.c.tgz 2023-10-18T18:07:39.293 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2023-10-18T18:07:39.300 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.yAn27A32Yk 2023-10-18T18:07:39.300 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.yAn27A32Yk 2023-10-18T18:07:39.570 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.yAn27A32Yk 2023-10-18T18:07:39.591 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.yAn27A32Yk is 951KB 2023-10-18T18:07:39.664 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.yAn27A32Yk 2023-10-18T18:07:39.672 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-18T18:07:39.672 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-18T18:07:39.677 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-18T18:07:39.713 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-18T18:07:39.714 DEBUG:teuthology.orchestra.run.smithi072:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-18T18:07:39.766 DEBUG:teuthology.orchestra.run.smithi179:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-18T18:07:39.791 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-18T18:07:39.822 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-18T18:07:39.837 INFO:tasks.ceph:Archiving crash dumps... 2023-10-18T18:07:39.839 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/remote/smithi072/crash 2023-10-18T18:07:39.840 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-18T18:07:39.890 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/remote/smithi179/crash 2023-10-18T18:07:39.891 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-18T18:07:39.922 INFO:tasks.ceph:Shutting down logrotate 2023-10-18T18:07:39.923 DEBUG:teuthology.orchestra.run.smithi072:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:07:39.940 DEBUG:teuthology.orchestra.run.smithi179:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T18:07:40.000 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-18T18:07:40.034 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-18T18:07:40.050 INFO:tasks.ceph:Compressing logs... 2023-10-18T18:07:40.051 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T18:07:40.087 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T18:07:40.118 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2023-10-18T18:07:40.119 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-18T18:07:40.120 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33912.log 2023-10-18T18:07:40.120 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-18T18:07:40.121 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-client.admin.33912.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-18T18:07:40.121 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-mon.b.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33912.log.gz 2023-10-18T18:07:40.122 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33939.log 2023-10-18T18:07:40.122 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-18T18:07:40.123 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.33939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33810.log 2023-10-18T18:07:40.124 INFO:teuthology.orchestra.run.smithi179.stderr: 88.1% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2023-10-18T18:07:40.124 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-mgr.x.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.33939.log.gz 2023-10-18T18:07:40.125 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40273.log 2023-10-18T18:07:40.125 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-18T18:07:40.126 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-client.admin.33810.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33810.log.gz 2023-10-18T18:07:40.126 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-18T18:07:40.126 INFO:teuthology.orchestra.run.smithi179.stderr: 89.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-18T18:07:40.127 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40001.log 2023-10-18T18:07:40.128 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-18T18:07:40.128 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33860.log 2023-10-18T18:07:40.129 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38607.log 2023-10-18T18:07:40.129 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40273.log: /var/log/ceph/ceph-client.admin.40001.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40273.log.gz 2023-10-18T18:07:40.130 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40001.log.gz 2023-10-18T18:07:40.130 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36972.log 2023-10-18T18:07:40.131 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38607.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38607.log.gz 2023-10-18T18:07:40.131 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39454.log 2023-10-18T18:07:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-18T18:07:40.132 INFO:teuthology.orchestra.run.smithi179.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-18T18:07:40.132 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-client.admin.33860.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-18T18:07:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-18T18:07:40.133 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mds.b.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.33860.log.gz 2023-10-18T18:07:40.134 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36807.log 2023-10-18T18:07:40.134 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36972.log.gz 2023-10-18T18:07:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33835.log 2023-10-18T18:07:40.136 INFO:teuthology.orchestra.run.smithi179.stderr: 93.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-18T18:07:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33968.log 2023-10-18T18:07:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.33835.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33835.log.gz 2023-10-18T18:07:40.136 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33887.log 2023-10-18T18:07:40.137 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-client.admin.33968.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33968.log.gz 2023-10-18T18:07:40.138 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39454.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-18T18:07:40.139 INFO:teuthology.orchestra.run.smithi072.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39454.log.gz 2023-10-18T18:07:40.139 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37628.log 2023-10-18T18:07:40.139 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36807.log.gz 2023-10-18T18:07:40.139 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39891.log 2023-10-18T18:07:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-18T18:07:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-client.admin.33887.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33887.log.gz 2023-10-18T18:07:40.140 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33995.log 2023-10-18T18:07:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-osd.7.log: 88.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-18T18:07:40.141 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/ceph-client.admin.33995.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33995.log.gz 2023-10-18T18:07:40.142 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37628.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37628.log.gz 2023-10-18T18:07:40.142 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38251.log 2023-10-18T18:07:40.143 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39891.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39891.log.gz 2023-10-18T18:07:40.143 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39373.log 2023-10-18T18:07:40.143 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38251.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38251.log.gz 2023-10-18T18:07:40.144 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37276.log 2023-10-18T18:07:40.144 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39373.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39373.log.gz 2023-10-18T18:07:40.145 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38799.log 2023-10-18T18:07:40.145 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37276.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37276.log.gz 2023-10-18T18:07:40.146 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37763.log 2023-10-18T18:07:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38799.log: gzip -5 87.4%--verbose -- replaced with /var/log/ceph/ceph-client.admin.38799.log.gz-- 2023-10-18T18:07:40.147 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39318.log 2023-10-18T18:07:40.148 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37763.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37763.log.gz 2023-10-18T18:07:40.148 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-18T18:07:40.150 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39318.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39318.log.gz 2023-10-18T18:07:40.150 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33227.log 2023-10-18T18:07:40.151 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38470.log 2023-10-18T18:07:40.152 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33227.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33227.log.gz 2023-10-18T18:07:40.153 INFO:teuthology.orchestra.run.smithi072.stderr: 88.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-18T18:07:40.153 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38907.log 2023-10-18T18:07:40.154 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38470.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38470.log.gz 2023-10-18T18:07:40.155 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37166.log 2023-10-18T18:07:40.156 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38988.log 2023-10-18T18:07:40.157 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38907.log: /var/log/ceph/ceph-client.admin.37166.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37166.log.gz 2023-10-18T18:07:40.157 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39972.log 2023-10-18T18:07:40.158 INFO:teuthology.orchestra.run.smithi072.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38907.log.gz 2023-10-18T18:07:40.159 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38988.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38988.log.gz 2023-10-18T18:07:40.159 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39208.log 2023-10-18T18:07:40.161 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36607.log 2023-10-18T18:07:40.161 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39972.log.gz 2023-10-18T18:07:40.162 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39208.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39208.log.gz 2023-10-18T18:07:40.163 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38853.log 2023-10-18T18:07:40.165 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36607.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36607.log.gz 2023-10-18T18:07:40.165 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-18T18:07:40.166 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38716.log 2023-10-18T18:07:40.166 INFO:teuthology.orchestra.run.smithi072.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38853.log.gz 2023-10-18T18:07:40.168 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39918.log 2023-10-18T18:07:40.168 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38716.log: 86.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-18T18:07:40.169 INFO:teuthology.orchestra.run.smithi072.stderr:gzip 87.5%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.38716.log.gz-- 2023-10-18T18:07:40.169 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40055.log 2023-10-18T18:07:40.170 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39918.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39918.log.gz 2023-10-18T18:07:40.170 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37952.log 2023-10-18T18:07:40.171 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40055.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40055.log.gz 2023-10-18T18:07:40.171 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39400.log 2023-10-18T18:07:40.172 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37979.log 2023-10-18T18:07:40.173 INFO:teuthology.orchestra.run.smithi072.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37952.log.gz 2023-10-18T18:07:40.173 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39400.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39400.log.gz 2023-10-18T18:07:40.174 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40082.log 2023-10-18T18:07:40.175 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37979.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37979.log.gz 2023-10-18T18:07:40.176 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39510.log 2023-10-18T18:07:40.176 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40082.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40082.log.gz 2023-10-18T18:07:40.177 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38770.log 2023-10-18T18:07:40.178 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37249.log 2023-10-18T18:07:40.178 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39510.log.gz 2023-10-18T18:07:40.179 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38770.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38770.log.gz 2023-10-18T18:07:40.180 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38170.log 2023-10-18T18:07:40.180 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37249.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37249.log.gz 2023-10-18T18:07:40.181 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39672.log 2023-10-18T18:07:40.183 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38170.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38170.log.gz 2023-10-18T18:07:40.183 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37655.log 2023-10-18T18:07:40.184 INFO:teuthology.orchestra.run.smithi179.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-18T18:07:40.186 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39672.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-18T18:07:40.186 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39672.log.gz 2023-10-18T18:07:40.187 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37655.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2023-10-18T18:07:40.187 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37655.log.gz 2023-10-18T18:07:40.188 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38416.log 2023-10-18T18:07:40.189 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39015.log 2023-10-18T18:07:40.190 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38416.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38416.log.gz 2023-10-18T18:07:40.191 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38006.log 2023-10-18T18:07:40.191 INFO:teuthology.orchestra.run.smithi072.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-18T18:07:40.192 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39015.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39015.log.gz 2023-10-18T18:07:40.193 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39618.log 2023-10-18T18:07:40.194 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38006.log: 89.0% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2023-10-18T18:07:40.194 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38006.log.gz 2023-10-18T18:07:40.194 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39072.log 2023-10-18T18:07:40.195 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39618.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39618.log.gz 2023-10-18T18:07:40.196 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39181.log 2023-10-18T18:07:40.196 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39072.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39072.log.gz 2023-10-18T18:07:40.196 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40300.log 2023-10-18T18:07:40.198 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37384.log 2023-10-18T18:07:40.198 INFO:teuthology.orchestra.run.smithi072.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39181.log.gz 2023-10-18T18:07:40.199 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38197.log 2023-10-18T18:07:40.200 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37384.log: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.40300.log.gz 2023-10-18T18:07:40.201 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37925.log 2023-10-18T18:07:40.201 INFO:teuthology.orchestra.run.smithi072.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37384.log.gz 2023-10-18T18:07:40.201 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38197.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38197.log.gz 2023-10-18T18:07:40.202 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37357.log 2023-10-18T18:07:40.203 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37844.log 2023-10-18T18:07:40.205 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37925.log: /var/log/ceph/ceph-client.admin.37357.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37357.log.gz 2023-10-18T18:07:40.205 INFO:teuthology.orchestra.run.smithi072.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37925.log.gz 2023-10-18T18:07:40.206 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37030.log 2023-10-18T18:07:40.207 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39235.log 2023-10-18T18:07:40.207 INFO:teuthology.orchestra.run.smithi072.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37844.log.gz 2023-10-18T18:07:40.208 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37030.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37030.log.gz 2023-10-18T18:07:40.209 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36837.log 2023-10-18T18:07:40.210 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39235.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39235.log.gz 2023-10-18T18:07:40.211 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37709.log 2023-10-18T18:07:40.212 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40136.log 2023-10-18T18:07:40.213 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36837.log: /var/log/ceph/ceph-client.admin.37709.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36837.log.gz 2023-10-18T18:07:40.214 INFO:teuthology.orchestra.run.smithi072.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37709.log.gz 2023-10-18T18:07:40.215 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38335.log 2023-10-18T18:07:40.215 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37222.log 2023-10-18T18:07:40.216 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40136.log.gz 2023-10-18T18:07:40.216 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38335.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38335.log.gz 2023-10-18T18:07:40.217 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39126.log 2023-10-18T18:07:40.220 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38689.log 2023-10-18T18:07:40.220 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37222.log.gz 2023-10-18T18:07:40.221 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38308.log 2023-10-18T18:07:40.222 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39126.log.gz 2023-10-18T18:07:40.222 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38689.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38689.log.gz 2023-10-18T18:07:40.223 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38826.log 2023-10-18T18:07:40.224 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38308.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38308.log.gz 2023-10-18T18:07:40.224 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-18T18:07:40.226 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38826.log: gzip -5 87.9%--verbose -- replaced with /var/log/ceph/ceph-client.admin.38826.log.gz-- 2023-10-18T18:07:40.226 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39264.log 2023-10-18T18:07:40.227 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37682.log 2023-10-18T18:07:40.228 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39264.log: 88.8% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39264.log.gz -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-18T18:07:40.228 INFO:teuthology.orchestra.run.smithi072.stderr: 2023-10-18T18:07:40.229 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37195.log 2023-10-18T18:07:40.230 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37682.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37682.log.gz 2023-10-18T18:07:40.231 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39810.log 2023-10-18T18:07:40.232 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38033.log 2023-10-18T18:07:40.232 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37195.log: /var/log/ceph/ceph-client.admin.39810.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39810.log.gz 2023-10-18T18:07:40.233 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37195.log.gz 2023-10-18T18:07:40.234 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36633.log 2023-10-18T18:07:40.235 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38033.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38033.log.gz 2023-10-18T18:07:40.236 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38497.log 2023-10-18T18:07:40.237 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39537.log 2023-10-18T18:07:40.238 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36633.log: /var/log/ceph/ceph-client.admin.38497.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38497.log.gz 2023-10-18T18:07:40.238 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36633.log.gz 2023-10-18T18:07:40.239 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38443.log 2023-10-18T18:07:40.240 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38389.log 2023-10-18T18:07:40.240 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38443.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39537.log.gz 2023-10-18T18:07:40.241 INFO:teuthology.orchestra.run.smithi072.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38443.log.gz 2023-10-18T18:07:40.241 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37520.log 2023-10-18T18:07:40.242 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40109.log 2023-10-18T18:07:40.243 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38389.log.gz 2023-10-18T18:07:40.243 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37520.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37520.log.gz 2023-10-18T18:07:40.244 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37438.log 2023-10-18T18:07:40.245 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39291.log 2023-10-18T18:07:40.245 INFO:teuthology.orchestra.run.smithi072.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40109.log.gz 2023-10-18T18:07:40.246 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37438.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37438.log.gz 2023-10-18T18:07:40.247 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38524.log 2023-10-18T18:07:40.248 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38580.log 2023-10-18T18:07:40.250 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39291.log: /var/log/ceph/ceph-client.admin.38524.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38524.log.gz 2023-10-18T18:07:40.250 INFO:teuthology.orchestra.run.smithi072.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39291.log.gz 2023-10-18T18:07:40.250 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37465.log 2023-10-18T18:07:40.251 INFO:teuthology.orchestra.run.smithi179.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-18T18:07:40.252 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37790.log 2023-10-18T18:07:40.252 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38580.log.gz 2023-10-18T18:07:40.252 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37465.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37465.log.gz 2023-10-18T18:07:40.253 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37303.log 2023-10-18T18:07:40.254 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39945.log 2023-10-18T18:07:40.255 INFO:teuthology.orchestra.run.smithi072.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37790.log.gz 2023-10-18T18:07:40.255 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37303.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37303.log.gz 2023-10-18T18:07:40.256 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38089.log 2023-10-18T18:07:40.258 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39945.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39945.log.gz 2023-10-18T18:07:40.258 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40246.log 2023-10-18T18:07:40.259 INFO:teuthology.orchestra.run.smithi179.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-18T18:07:40.260 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40190.log 2023-10-18T18:07:40.261 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38089.log: /var/log/ceph/ceph-client.admin.40246.log: gzip 87.5%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.38089.log.gz-- /var/log/ceph/ceph-client.admin.39427.log 2023-10-18T18:07:40.261 INFO:teuthology.orchestra.run.smithi072.stderr: 2023-10-18T18:07:40.261 INFO:teuthology.orchestra.run.smithi072.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40246.log.gz 2023-10-18T18:07:40.262 INFO:teuthology.orchestra.run.smithi179.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-18T18:07:40.262 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40190.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40190.log.gz 2023-10-18T18:07:40.263 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33103.log 2023-10-18T18:07:40.264 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38553.log 2023-10-18T18:07:40.265 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39427.log: /var/log/ceph/ceph-client.admin.33103.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33103.log.gz 2023-10-18T18:07:40.265 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39481.log 2023-10-18T18:07:40.266 INFO:teuthology.orchestra.run.smithi072.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39427.log.gz 2023-10-18T18:07:40.266 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36945.log 2023-10-18T18:07:40.267 INFO:teuthology.orchestra.run.smithi072.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38553.log.gz 2023-10-18T18:07:40.267 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39481.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39481.log.gz 2023-10-18T18:07:40.268 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37492.log 2023-10-18T18:07:40.269 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36945.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36945.log.gz 2023-10-18T18:07:40.269 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37871.log 2023-10-18T18:07:40.270 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37492.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37492.log.gz 2023-10-18T18:07:40.270 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37547.log 2023-10-18T18:07:40.271 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37871.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37871.log.gz 2023-10-18T18:07:40.271 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37411.log 2023-10-18T18:07:40.272 INFO:teuthology.orchestra.run.smithi179.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-18T18:07:40.273 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37547.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-18T18:07:40.273 INFO:teuthology.orchestra.run.smithi072.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37547.log.gz 2023-10-18T18:07:40.273 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37411.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37411.log.gz 2023-10-18T18:07:40.274 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39699.log 2023-10-18T18:07:40.274 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37601.log 2023-10-18T18:07:40.275 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39699.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39699.log.gz 2023-10-18T18:07:40.276 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38116.log 2023-10-18T18:07:40.277 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39591.log 2023-10-18T18:07:40.278 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38116.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37601.log.gz 2023-10-18T18:07:40.278 INFO:teuthology.orchestra.run.smithi072.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38116.log.gz 2023-10-18T18:07:40.279 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39864.log 2023-10-18T18:07:40.281 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38634.log 2023-10-18T18:07:40.281 INFO:teuthology.orchestra.run.smithi072.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39591.log.gz 2023-10-18T18:07:40.282 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39864.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39864.log.gz 2023-10-18T18:07:40.283 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36864.log 2023-10-18T18:07:40.283 INFO:teuthology.orchestra.run.smithi179.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-18T18:07:40.284 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36660.log 2023-10-18T18:07:40.285 INFO:teuthology.orchestra.run.smithi072.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38634.log.gz 2023-10-18T18:07:40.285 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36864.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36864.log.gz 2023-10-18T18:07:40.286 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38743.log 2023-10-18T18:07:40.287 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36660.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36660.log.gz 2023-10-18T18:07:40.287 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37139.log 2023-10-18T18:07:40.288 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37898.log 2023-10-18T18:07:40.288 INFO:teuthology.orchestra.run.smithi072.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38743.log.gz 2023-10-18T18:07:40.289 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37139.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28219.log 2023-10-18T18:07:40.290 INFO:teuthology.orchestra.run.smithi072.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37139.log.gz 2023-10-18T18:07:40.290 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39346.log 2023-10-18T18:07:40.291 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37898.log.gz 2023-10-18T18:07:40.291 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.tmp-client.admin.28219.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28219.log.gz 2023-10-18T18:07:40.292 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33354.log 2023-10-18T18:07:40.292 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40028.log 2023-10-18T18:07:40.293 INFO:teuthology.orchestra.run.smithi072.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39346.log.gz 2023-10-18T18:07:40.294 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37330.log 2023-10-18T18:07:40.294 INFO:teuthology.orchestra.run.smithi072.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33354.log.gz 2023-10-18T18:07:40.295 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40028.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40028.log.gz 2023-10-18T18:07:40.295 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40217.log 2023-10-18T18:07:40.296 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37330.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37330.log.gz 2023-10-18T18:07:40.296 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39837.log 2023-10-18T18:07:40.297 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37112.log 2023-10-18T18:07:40.298 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40217.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40217.log.gz 2023-10-18T18:07:40.298 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39837.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39837.log.gz 2023-10-18T18:07:40.299 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37574.log 2023-10-18T18:07:40.299 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37112.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37112.log.gz 2023-10-18T18:07:40.300 INFO:teuthology.orchestra.run.smithi072.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.39099.log 2023-10-18T18:07:40.301 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38880.log 2023-10-18T18:07:40.301 INFO:teuthology.orchestra.run.smithi072.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37574.log.gz 2023-10-18T18:07:40.302 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39645.log 2023-10-18T18:07:40.302 INFO:teuthology.orchestra.run.smithi072.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39099.log.gz 2023-10-18T18:07:40.303 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38880.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38880.log.gz 2023-10-18T18:07:40.303 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37058.log 2023-10-18T18:07:40.304 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37817.log 2023-10-18T18:07:40.305 INFO:teuthology.orchestra.run.smithi072.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39645.log.gz 2023-10-18T18:07:40.306 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37058.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37058.log.gz 2023-10-18T18:07:40.306 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39044.log 2023-10-18T18:07:40.306 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38278.log 2023-10-18T18:07:40.307 INFO:teuthology.orchestra.run.smithi072.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37817.log.gz 2023-10-18T18:07:40.308 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39044.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39044.log.gz 2023-10-18T18:07:40.308 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38934.log 2023-10-18T18:07:40.309 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40327.log 2023-10-18T18:07:40.309 INFO:teuthology.orchestra.run.smithi072.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38278.log.gz 2023-10-18T18:07:40.310 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38934.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38934.log.gz 2023-10-18T18:07:40.310 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39783.log 2023-10-18T18:07:40.311 INFO:teuthology.orchestra.run.smithi179.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-18T18:07:40.312 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40327.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-18T18:07:40.313 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39783.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39783.log.gz 2023-10-18T18:07:40.313 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-18T18:07:40.314 INFO:teuthology.orchestra.run.smithi179.stderr: 2023-10-18T18:07:40.314 INFO:teuthology.orchestra.run.smithi179.stderr:real 0m0.219s 2023-10-18T18:07:40.314 INFO:teuthology.orchestra.run.smithi179.stderr:user 0m0.946s 2023-10-18T18:07:40.314 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m0.055s 2023-10-18T18:07:40.315 INFO:teuthology.orchestra.run.smithi072.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.40327.log.gz 2023-10-18T18:07:40.315 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38362.log 2023-10-18T18:07:40.316 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.0.log: 87.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-18T18:07:40.316 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38062.log 2023-10-18T18:07:40.316 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38362.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38362.log.gz 2023-10-18T18:07:40.317 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36918.log 2023-10-18T18:07:40.318 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38062.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38062.log.gz 2023-10-18T18:07:40.319 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39726.log 2023-10-18T18:07:40.320 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40163.log 2023-10-18T18:07:40.320 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36918.log.gz 2023-10-18T18:07:40.322 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38143.log 2023-10-18T18:07:40.322 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39726.log.gz 2023-10-18T18:07:40.324 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39564.log 2023-10-18T18:07:40.324 INFO:teuthology.orchestra.run.smithi072.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40163.log.gz 2023-10-18T18:07:40.325 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38143.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-18T18:07:40.325 INFO:teuthology.orchestra.run.smithi072.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38143.log.gz 2023-10-18T18:07:40.327 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39564.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39564.log.gz 2023-10-18T18:07:40.327 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39755.log 2023-10-18T18:07:40.329 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36696.log 2023-10-18T18:07:40.330 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36780.log 2023-10-18T18:07:40.330 INFO:teuthology.orchestra.run.smithi072.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39755.log.gz 2023-10-18T18:07:40.332 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38662.log 2023-10-18T18:07:40.332 INFO:teuthology.orchestra.run.smithi072.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36696.log.gz 2023-10-18T18:07:40.333 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37085.log 2023-10-18T18:07:40.333 INFO:teuthology.orchestra.run.smithi072.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36780.log.gz 2023-10-18T18:07:40.335 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39154.log 2023-10-18T18:07:40.335 INFO:teuthology.orchestra.run.smithi072.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38662.log.gz 2023-10-18T18:07:40.336 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36891.log 2023-10-18T18:07:40.336 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37085.log.gz 2023-10-18T18:07:40.338 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37736.log 2023-10-18T18:07:40.338 INFO:teuthology.orchestra.run.smithi072.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39154.log.gz 2023-10-18T18:07:40.339 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36891.log: gzip -5 88.1%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.36891.log.gz/var/log/ceph/ceph-client.admin.38224.log 2023-10-18T18:07:40.339 INFO:teuthology.orchestra.run.smithi072.stderr: 2023-10-18T18:07:40.341 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37001.log 2023-10-18T18:07:40.341 INFO:teuthology.orchestra.run.smithi072.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37736.log.gz 2023-10-18T18:07:40.342 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38224.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-18T18:07:40.343 INFO:teuthology.orchestra.run.smithi072.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38224.log.gz 2023-10-18T18:07:40.344 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38961.log 2023-10-18T18:07:40.344 INFO:teuthology.orchestra.run.smithi072.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37001.log.gz 2023-10-18T18:07:40.346 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.38961.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38961.log.gz 2023-10-18T18:07:40.362 INFO:teuthology.orchestra.run.smithi072.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-18T18:07:40.400 INFO:teuthology.orchestra.run.smithi072.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-18T18:07:40.461 INFO:teuthology.orchestra.run.smithi072.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-18T18:07:40.466 INFO:teuthology.orchestra.run.smithi072.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-18T18:07:40.470 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-18T18:07:40.473 INFO:teuthology.orchestra.run.smithi072.stderr: 2023-10-18T18:07:40.474 INFO:teuthology.orchestra.run.smithi072.stderr:real 0m0.378s 2023-10-18T18:07:40.474 INFO:teuthology.orchestra.run.smithi072.stderr:user 0m0.882s 2023-10-18T18:07:40.474 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m0.422s 2023-10-18T18:07:40.475 INFO:tasks.ceph:Archiving logs... 2023-10-18T18:07:40.476 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/remote/smithi072/log 2023-10-18T18:07:40.477 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T18:07:40.780 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/remote/smithi179/log 2023-10-18T18:07:40.782 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T18:07:40.908 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2023-10-18T18:07:40.983 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=cc911913c4a742c194159c61847c1b8e Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1905, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1482, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3166, in wait_until_healthy while self.get_mon_health()['status'] != 'HEALTH_OK': File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 3158, in get_mon_health out = self.raw_cluster_cmd('health', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph.py", line 1911, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2286, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 2520, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1600, in raw_cluster_cmd p = self.run_cluster_cmd(args=args, stdout=stdout, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_b4ddd52ca4ac2ca6dc3b66adb82565f632d0f5a5/qa/tasks/ceph_manager.py", line 1593, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2023-10-18T18:07:40.990 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-18T18:07:41.012 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-18T18:07:41.042 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-18T18:07:41.042 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T18:07:41.056 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T18:07:41.131 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-18T18:07:41.140 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-18T18:07:41.140 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T18:07:41.148 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T18:07:41.173 INFO:teuthology.orchestra.run.smithi179.stdout: remote refid st t when poll reach delay offset jitter 2023-10-18T18:07:41.173 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================== 2023-10-18T18:07:41.173 INFO:teuthology.orchestra.run.smithi179.stdout:+hv01.front.sepi 67.205.162.81 3 u 13 64 377 0.091 -2.313 0.053 2023-10-18T18:07:41.174 INFO:teuthology.orchestra.run.smithi179.stdout:+hv02.front.sepi 63.231.80.2 3 u 17 64 377 0.089 2.655 0.044 2023-10-18T18:07:41.174 INFO:teuthology.orchestra.run.smithi179.stdout:*hv03.front.sepi 74.6.168.72 3 u 7 64 377 0.117 1.238 0.035 2023-10-18T18:07:41.174 INFO:teuthology.orchestra.run.smithi179.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 0.000 0.000 2023-10-18T18:07:41.182 INFO:teuthology.orchestra.run.smithi072.stdout: remote refid st t when poll reach delay offset jitter 2023-10-18T18:07:41.182 INFO:teuthology.orchestra.run.smithi072.stdout:============================================================================== 2023-10-18T18:07:41.183 INFO:teuthology.orchestra.run.smithi072.stdout:+hv01.front.sepi 67.205.162.81 3 u 14 64 377 0.120 -1.873 0.312 2023-10-18T18:07:41.183 INFO:teuthology.orchestra.run.smithi072.stdout:+hv02.front.sepi 63.231.80.2 3 u 9 64 377 0.117 3.078 0.380 2023-10-18T18:07:41.183 INFO:teuthology.orchestra.run.smithi072.stdout:*hv03.front.sepi 74.6.168.72 3 u 1 64 377 0.099 2.245 0.421 2023-10-18T18:07:41.183 INFO:teuthology.orchestra.run.smithi072.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 0.000 0.000 2023-10-18T18:07:41.185 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-18T18:07:41.194 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-18T18:07:41.194 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-18T18:07:41.285 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-18T18:07:41.313 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-18T18:07:41.450 INFO:teuthology.task.internal:Duration was 1283.674796 seconds 2023-10-18T18:07:41.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-18T18:07:41.491 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-18T18:07:41.491 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T18:07:41.499 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T18:07:41.632 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-18T18:07:41.633 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2023-10-18T18:07:41.633 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T18:07:41.665 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2023-10-18T18:07:41.665 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T18:07:41.681 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-18T18:07:41.682 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T18:07:41.719 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T18:07:41.773 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-18T18:07:41.773 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T18:07:41.778 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T18:07:41.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-18T18:07:41.881 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-18T18:07:41.881 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T18:07:41.917 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T18:07:41.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-18T18:07:41.944 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T18:07:41.967 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T18:07:41.981 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2023-10-18T18:07:41.998 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2023-10-18T18:07:42.088 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T18:07:42.150 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi072.front.sepia.ceph.com, flagging run as failed 2023-10-18T18:07:42.151 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T18:07:42.157 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi179.front.sepia.ceph.com, flagging run as failed 2023-10-18T18:07:42.158 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-18T18:07:42.169 INFO:teuthology.task.internal:Transferring archived files... 2023-10-18T18:07:42.170 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/remote/smithi072 2023-10-18T18:07:42.170 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T18:08:06.714 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2023-10-18T18:08:06.725 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161/remote/smithi072/coredump/1697652009.36699.core.gz: gzip compressed data, was "1697652009.36699.core", last modified: Wed Oct 18 18:07:50 2023, max compression, original size modulo 2^32 299315200 2023-10-18T18:08:06.726 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2023-10-18T18:08:07.909 INFO:teuthology.task.internal: core looks like: /tmp/tmpwh7x3nz3: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i e', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/bin/ceph-mds', platform: 'x86_64' 2023-10-18T18:08:07.942 INFO:teuthology.task.internal: dump_program: ceph-mds -f --cluster ceph -i e 2023-10-18T18:08:07.942 DEBUG:teuthology.orchestra.run.smithi072:> which 'ceph-mds -f --cluster ceph -i e' 2023-10-18T18:08:07.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T18:08:07.963 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 1: "which 'ceph-mds -f --cluster ceph -i e'" 2023-10-18T18:08:07.970 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-18T18:08:07.981 INFO:teuthology.task.internal:Not uploading archives. 2023-10-18T18:08:07.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-18T18:08:08.020 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-18T18:08:08.021 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T18:08:08.031 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T18:08:08.040 INFO:teuthology.orchestra.run.smithi179.stdout: 1048603 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 18 18:07 /home/ubuntu/cephtest 2023-10-18T18:08:08.041 INFO:teuthology.orchestra.run.smithi179.stdout: 1048620 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Oct 18 17:46 /home/ubuntu/cephtest/archive 2023-10-18T18:08:08.041 INFO:teuthology.orchestra.run.smithi179.stdout: 1048622 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 18:07 /home/ubuntu/cephtest/archive/syslog 2023-10-18T18:08:08.041 INFO:teuthology.orchestra.run.smithi179.stdout: 1048618 4 -rw-rw-rw- 1 ubuntu ubuntu 570 Oct 18 18:07 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-10-18T18:08:08.041 INFO:teuthology.orchestra.run.smithi179.stdout: 1048623 84 -rw-rw-rw- 1 ubuntu ubuntu 83053 Oct 18 18:07 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-10-18T18:08:08.042 INFO:teuthology.orchestra.run.smithi179.stdout: 1048624 96 -rw-rw-r-- 1 ubuntu ubuntu 96913 Oct 18 18:07 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T18:08:08.042 INFO:teuthology.orchestra.run.smithi179.stdout: 1048621 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 17:59 /home/ubuntu/cephtest/archive/coredump 2023-10-18T18:08:08.042 INFO:teuthology.orchestra.run.smithi179.stdout: 1050539 13204 -rw------- 1 root root 299315200 Oct 18 17:59 /home/ubuntu/cephtest/archive/coredump/1697651979.37436.core 2023-10-18T18:08:08.043 INFO:teuthology.orchestra.run.smithi179.stdout: 1050538 12564 -rw------- 1 root root 299311104 Oct 18 17:59 /home/ubuntu/cephtest/archive/coredump/1697651949.37437.core 2023-10-18T18:08:08.043 INFO:teuthology.orchestra.run.smithi179.stdout: 1050540 13536 -rw------- 1 root root 299315200 Oct 18 17:59 /home/ubuntu/cephtest/archive/coredump/1697651994.37438.core 2023-10-18T18:08:08.044 INFO:teuthology.orchestra.run.smithi179.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-10-18T18:08:08.049 INFO:teuthology.orchestra.run.smithi072.stdout: 1048592 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 18 18:07 /home/ubuntu/cephtest 2023-10-18T18:08:08.049 INFO:teuthology.orchestra.run.smithi072.stdout: 1048593 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Oct 18 17:46 /home/ubuntu/cephtest/archive 2023-10-18T18:08:08.049 INFO:teuthology.orchestra.run.smithi072.stdout: 1053530 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 18:07 /home/ubuntu/cephtest/archive/syslog 2023-10-18T18:08:08.050 INFO:teuthology.orchestra.run.smithi072.stdout: 1048599 4 -rw-rw-rw- 1 ubuntu ubuntu 2430 Oct 18 18:07 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-10-18T18:08:08.050 INFO:teuthology.orchestra.run.smithi072.stdout: 1048600 88 -rw-rw-rw- 1 ubuntu ubuntu 88432 Oct 18 18:07 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-10-18T18:08:08.050 INFO:teuthology.orchestra.run.smithi072.stdout: 1048611 100 -rw-rw-r-- 1 ubuntu ubuntu 100376 Oct 18 18:07 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T18:08:08.051 INFO:teuthology.orchestra.run.smithi072.stdout: 1048594 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 18:00 /home/ubuntu/cephtest/archive/coredump 2023-10-18T18:08:08.051 INFO:teuthology.orchestra.run.smithi072.stdout: 1048609 14536 -rw------- 1 root root 299319296 Oct 18 17:59 /home/ubuntu/cephtest/archive/coredump/1697651964.36698.core 2023-10-18T18:08:08.051 INFO:teuthology.orchestra.run.smithi072.stdout: 1048610 14268 -rw------- 1 root root 299315200 Oct 18 18:00 /home/ubuntu/cephtest/archive/coredump/1697652009.36699.core 2023-10-18T18:08:08.051 INFO:teuthology.orchestra.run.smithi072.stdout: 1048608 15664 -rw------- 1 root root 299315200 Oct 18 17:58 /home/ubuntu/cephtest/archive/coredump/1697651903.36697.core 2023-10-18T18:08:08.054 INFO:teuthology.orchestra.run.smithi072.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-10-18T18:08:08.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T18:08:08.056 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 1: "which 'ceph-mds -f --cluster ceph -i e'" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi072 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-10-18T18:08:08.064 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-18T18:08:08.074 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-18T18:08:08.084 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi072 with status 1: "which 'ceph-mds -f --cluster ceph -i e'" 2023-10-18T18:08:08.085 INFO:teuthology.nuke:Checking targets against current locks 2023-10-18T18:08:08.114 DEBUG:teuthology.nuke:shortname: smithi072 2023-10-18T18:08:08.115 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T18:08:08.137 DEBUG:teuthology.nuke:shortname: smithi179 2023-10-18T18:08:08.138 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T18:08:08.141 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 17:34:57.932941', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-18T18:08:08.159 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-16_12:48:28-fs-wip-vshankar-testing-pacific-20231016.091648-testing-default-smithi/7429161', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 17:34:57.934678', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-18T18:08:08.194 INFO:teuthology.orchestra.console.smithi072:Power off 2023-10-18T18:08:08.195 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T18:08:08.212 INFO:teuthology.orchestra.console.smithi179:Power off 2023-10-18T18:08:08.212 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T18:08:08.221 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2023-10-18T18:08:08.221 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T18:08:08.235 DEBUG:teuthology.orchestra.console.smithi179:power off output: Chassis Power Control: Down/Off 2023-10-18T18:08:08.236 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T18:08:08.246 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2023-10-18T18:08:08.259 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2023-10-18T18:08:12.249 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T18:08:12.260 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T18:08:12.360 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2023-10-18T18:08:12.371 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2023-10-18T18:08:16.362 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T18:08:16.373 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T18:08:16.472 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2023-10-18T18:08:16.485 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2023-10-18T18:08:16.574 INFO:teuthology.orchestra.console.smithi072:Power off completed 2023-10-18T18:08:20.489 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T18:08:20.598 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off 2023-10-18T18:08:20.700 INFO:teuthology.orchestra.console.smithi179:Power off completed 2023-10-18T18:08:20.805 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 description: fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}} duration: 1283.674795627594 failure_reason: 'Command failed on smithi072 with status 124: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=cc911913c4a742c194159c61847c1b8e status: fail success: false 2023-10-18T18:08:20.806 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T18:08:20.890 INFO:teuthology.run:FAIL